12:12:16,775 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:17,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7786059266674214771] 12:12:17,364 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:12:17,531 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:12:17,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:12:17,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:17,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:17,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:12:17,986 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:17,991 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:17,997 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:12:18,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:12:18,903 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 12:12:19,32 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,45 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:12:19,99 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:12:19,104 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 12:12:19,119 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 12:12:19,148 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 12:12:19,153 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:19,154 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 12:12:19,154 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 12:12:19,193 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1ad199f4 12:12:19,196 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 12:12:19,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:19,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 12:12:19,203 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 12:12:19,203 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 12:12:19,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-146771245] 12:12:19,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 12:12:19,205 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 12:12:19,205 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 12:12:19,205 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-726196697] 12:12:19,206 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-146771245] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-726196697] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,222 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-config_testBackupDatastore, persistent : true 12:12:19,227 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testBackupDatastore, persistent : true 12:12:19,253 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] 12:12:19,275 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:19,276 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 12:12:19,327 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] 12:12:19,328 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:19,328 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 12:12:19,355 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1906568960] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,356 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-1083803213] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,356 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-1083803213 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 12:12:19,356 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1906568960 created and ready for shard:member-1-shard-people-config_testBackupDatastore 12:12:19,364 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-1359076581] 12:12:19,365 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#98375685] 12:12:19,365 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-1359076581] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,365 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#98375685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,371 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testBackupDatastore with journal batch size 1 12:12:19,371 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-config_testBackupDatastore with journal batch size 1 12:12:19,378 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] 12:12:19,379 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:19,379 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 12:12:19,393 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:19,393 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:19,398 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:19,399 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:19,399 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:12:19,399 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 12:12:19,400 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:12:19,400 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 12:12:19,408 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,408 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 12:12:19,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 12:12:19,409 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 12:12:19,409 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 12:12:19,416 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 12:12:19,416 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 12:12:19,419 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 12:12:19,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 12:12:19,419 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-config_testBackupDatastore 12:12:19,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-config_testBackupDatastore 12:12:19,423 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 12:12:19,423 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 12:12:19,423 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:19,423 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:19,423 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,423 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,424 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 12:12:19,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 12:12:19,424 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 12:12:19,424 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 12:12:19,429 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] 12:12:19,430 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:19,430 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-146771245] 12:12:19,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Received ElectionTimeout 12:12:19,443 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Received ElectionTimeout 12:12:19,451 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 12:12:19,451 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 12:12:19,451 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:19,451 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:19,451 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 12:12:19,451 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 12:12:19,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 12:12:19,453 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 12:12:19,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 12:12:19,453 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 12:12:19,470 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-config_testBackupDatastore 12:12:19,470 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-config_testBackupDatastore 12:12:19,471 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testBackupDatastore, leaderId=member-1-shard-cars-config_testBackupDatastore, leaderPayloadVersion=13] 12:12:19,475 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testBackupDatastore, leaderId=member-1-shard-people-config_testBackupDatastore, leaderPayloadVersion=13] 12:12:19,476 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 12:12:19,476 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 12:12:19,476 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 12:12:19,520 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] 12:12:19,521 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:12:19,521 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-726196697] 12:12:19,522 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:19,527 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:19,532 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 12:12:19,535 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#-740009746] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,536 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:12:19,538 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:12:19,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 12:12:19,539 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 12:12:19,539 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:19,539 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 12:12:19,539 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 12:12:19,543 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 12:12:19,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d7956e0 12:12:19,544 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 12:12:19,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:19,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 12:12:19,546 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-oper_testBackupDatastore, persistent : true 12:12:19,547 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1835679701] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,547 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-783974714 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 12:12:19,547 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/shard-member-1:snapshot-read#1004053977] 12:12:19,548 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testBackupDatastore with journal batch size 1 12:12:19,548 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 12:12:19,548 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 12:12:19,548 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1835679701] 12:12:19,548 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 12:12:19,549 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 12:12:19,549 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 12:12:19,549 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#1328220718] 12:12:19,550 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testBackupDatastore, persistent : true 12:12:19,550 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-865531068 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 12:12:19,551 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-783974714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:19,551 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/shard-member-1:snapshot-read#2050409009] 12:12:19,551 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testBackupDatastore with journal batch size 1 12:12:19,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:19,552 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:19,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:12:19,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 12:12:19,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 12:12:19,553 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testBackupDatastore 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,553 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 12:12:19,553 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 12:12:19,554 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Received ElectionTimeout 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:19,555 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,555 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 12:12:19,555 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 12:12:19,555 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-oper_testBackupDatastore 12:12:19,556 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 12:12:19,556 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:19,556 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 12:12:19,556 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 12:12:19,556 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 12:12:19,555 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 12:12:19,556 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:19,557 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 12:12:19,557 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 12:12:19,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 12:12:19,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-oper_testBackupDatastore 12:12:19,557 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Received ElectionTimeout 12:12:19,557 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 12:12:19,557 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testBackupDatastore, leaderId=member-1-shard-cars-oper_testBackupDatastore, leaderPayloadVersion=13] 12:12:19,557 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:19,557 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 12:12:19,558 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 12:12:19,558 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 12:12:19,558 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 12:12:19,558 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-oper_testBackupDatastore 12:12:19,558 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testBackupDatastore, leaderId=member-1-shard-people-oper_testBackupDatastore, leaderPayloadVersion=13] 12:12:19,558 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 12:12:19,558 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 12:12:19,602 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#-740009746] 12:12:19,602 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:12:19,603 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1835679701] 12:12:19,653 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#-740009746] 12:12:19,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:12:19,654 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#1328220718] 12:12:19,673 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 12:12:19,674 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 12:12:19,677 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onGetSnapshot 12:12:19,678 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,678 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 12:12:19,680 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 12:12:19,680 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,695 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 12:12:19,696 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,696 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 12:12:19,698 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 12:12:19,698 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,699 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,702 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 12:12:19,702 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,702 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,704 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,701 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6f766767, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,701 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6da15dbb, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,715 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@112b4639, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,715 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 12:12:19,716 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6eb77d53, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,716 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 12:12:19,760 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 12:12:19,790 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@3159ae43 12:12:19,791 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#979703457] 12:12:19,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:19,791 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-146771245] 12:12:19,822 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testBackupDatastore 12:12:19,822 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Aborting 0 pending queued transactions 12:12:19,828 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 12:12:19,828 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 12:12:19,830 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 12:12:19,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 12:12:19,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 12:12:19,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 12:12:19,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,831 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 12:12:19,832 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:19,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,832 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 12:12:19,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,832 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@10d92739, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,832 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:19,832 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3a24b9a4, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,833 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7fd51dc3, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 12:12:19,833 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 12:12:20,49 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 12:12:20,51 PM [cluster-test-akka.actor.default-dispatcher-15] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:332) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 12:12:20,59 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 12:12:20,64 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 12:12:20,66 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 12:12:20,66 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onShutDown 12:12:20,68 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@33a7561d 12:12:20,68 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@33a7561d 12:12:20,69 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-config_testBackupDatastore 12:12:20,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Aborting 0 pending queued transactions 12:12:20,69 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:12:20,71 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:12:20,71 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:12:20,73 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 12:12:20,73 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 12:12:20,73 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 12:12:20,73 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:12:20,73 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onShutDown 12:12:20,74 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5cda62ff 12:12:20,74 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5cda62ff 12:12:20,74 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onShutDown 12:12:20,74 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5ac81c3c 12:12:20,74 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5ac81c3c 12:12:20,76 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testBackupDatastore 12:12:20,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Aborting 0 pending queued transactions 12:12:20,76 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testBackupDatastore 12:12:20,76 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Aborting 0 pending queued transactions 12:12:20,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:12:20,77 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:12:20,82 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:20,83 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:20,118 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:12:20,164 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:20,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4295585425911994548] 12:12:20,192 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:12:20,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:12:20,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:12:20,198 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:20,199 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:20,225 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,225 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:12:20,226 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:12:20,226 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:12:20,230 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,234 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 12:12:20,244 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1889544991] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,245 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:12:20,251 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:12:20,251 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,252 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,253 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:20,253 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,253 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,255 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 12:12:20,256 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2439ee3e 12:12:20,256 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 12:12:20,257 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:20,257 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,258 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,258 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,258 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-106203655] 12:12:20,258 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,259 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,259 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,259 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#1676438525] 12:12:20,260 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddReplicasForAllShards, persistent : true 12:12:20,260 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-106203655] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-1196139725 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1444504612] 12:12:20,261 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-1196139725] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,261 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testAddReplicasForAllShards, persistent : true 12:12:20,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1444504612] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#1676438525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,262 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#283477036] 12:12:20,263 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#269535798 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,263 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#269535798] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,265 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 12:12:20,267 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:20,267 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:20,268 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testAddReplicasForAllShards with journal batch size 1 12:12:20,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:12:20,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 12:12:20,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:20,270 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:20,270 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#283477036] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 12:12:20,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 12:12:20,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 12:12:20,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddReplicasForAllShards 12:12:20,271 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 12:12:20,272 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:20,272 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,272 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,273 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 12:12:20,273 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testAddReplicasForAllShards 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 12:12:20,273 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:20,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 12:12:20,274 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 12:12:20,274 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 12:12:20,274 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 12:12:20,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 12:12:20,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 12:12:20,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 12:12:20,275 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:20,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:20,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 12:12:20,275 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 12:12:20,276 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 12:12:20,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 12:12:20,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 12:12:20,276 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:20,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:20,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 12:12:20,276 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 12:12:20,276 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 12:12:20,276 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:20,276 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 12:12:20,277 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:20,277 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 12:12:20,277 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:20,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1889544991] 12:12:20,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:20,308 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-106203655] 12:12:20,359 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1889544991] 12:12:20,359 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:12:20,359 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#1676438525] 12:12:20,360 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,364 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,370 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 12:12:20,371 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-392379958] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,372 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:12:20,373 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:12:20,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:20,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,378 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 12:12:20,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3062ca1 12:12:20,380 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 12:12:20,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:20,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,381 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-196544683] 12:12:20,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,382 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddReplicasForAllShards, persistent : true 12:12:20,382 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-196544683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,383 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-people-oper_testAddReplicasForAllShards, persistent : true 12:12:20,384 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 12:12:20,384 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#1964069048 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,385 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#739083935] 12:12:20,385 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1762564991] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,386 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:20,388 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:20,388 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,388 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 12:12:20,388 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 12:12:20,388 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 12:12:20,388 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,389 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 12:12:20,389 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:20,389 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,389 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 12:12:20,389 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 12:12:20,390 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 12:12:20,388 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 12:12:20,387 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1023212695] 12:12:20,387 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1796598736 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,390 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 12:12:20,390 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 12:12:20,390 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:20,390 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:20,390 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 12:12:20,391 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 12:12:20,386 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 12:12:20,391 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 12:12:20,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,392 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 12:12:20,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 12:12:20,393 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 12:12:20,393 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 12:12:20,393 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:20,393 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:20,393 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 12:12:20,393 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 12:12:20,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1762564991] 12:12:20,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:12:20,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 12:12:20,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:12:20,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 12:12:20,394 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 12:12:20,394 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 12:12:20,394 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:20,394 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 12:12:20,394 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:20,394 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 12:12:20,394 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 12:12:20,436 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-392379958] 12:12:20,437 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:12:20,437 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-196544683] 12:12:20,488 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-392379958] 12:12:20,488 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:12:20,488 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1762564991] 12:12:20,503 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@61b54a5e] 12:12:20,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 12:12:20,510 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-pets-config_testAddReplicasForAllShards, persistent : true 12:12:20,511 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 12:12:20,512 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#269348502] 12:12:20,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-1345397093 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 12:12:20,513 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:20,513 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 12:12:20,514 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-pets-config_testAddReplicasForAllShards 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:20,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 12:12:20,515 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 12:12:20,515 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 12:12:20,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 12:12:20,516 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:20,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:20,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 12:12:20,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 12:12:20,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 12:12:20,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 12:12:20,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 12:12:20,517 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:20,517 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 12:12:20,517 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:20,567 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1889544991] 12:12:20,568 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 12:12:20,568 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-2022719567] 12:12:20,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:20,619 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3444717659799583648] 12:12:20,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:12:20,626 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:20,626 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:12:20,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:20,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:20,652 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,655 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,658 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 12:12:20,676 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#367713322] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:12:20,683 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:12:20,684 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 12:12:20,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@71e41837 12:12:20,689 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 12:12:20,690 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:20,690 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,693 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:20,696 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 12:12:20,698 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1788169108] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:20,698 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:12:20,699 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:12:20,704 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 12:12:20,704 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7b1c7347 12:12:20,705 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 12:12:20,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:21,3 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:12:21,56 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:21,63 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:21,63 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 12:12:21,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:21,63 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:21,212 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:12:21,213 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:21,213 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:21,213 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,213 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,213 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:21,213 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 12:12:21,214 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 12:12:21,214 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:21,316 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@227d91a1] 12:12:21,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 12:12:21,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@2339152d] 12:12:21,322 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-pets-config_testAddReplicasForAllShards, persistent : true 12:12:21,322 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 12:12:21,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#909739147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,323 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-816052709] 12:12:21,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-1543898817 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,323 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 12:12:21,324 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-1543898817] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,325 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-no-leader-oper_testAddReplicasForAllShards, persistent : true 12:12:21,325 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-816052709] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,325 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#830743676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,326 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-no-leader-oper_testAddReplicasForAllShards with journal batch size 1 12:12:21,326 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:21,326 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:21,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 12:12:21,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 12:12:21,327 PM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 12:12:21,327 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#569485209 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 12:12:21,327 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 12:12:21,328 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:21,328 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#569485209] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,328 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:21,330 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#1837939094] 12:12:21,330 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:21,330 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 12:12:21,330 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 12:12:21,330 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 12:12:21,332 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 12:12:21,332 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 12:12:21,333 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 12:12:21,334 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 12:12:21,334 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 12:12:21,334 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:21,335 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 12:12:21,338 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#1837939094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,339 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 12:12:21,340 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:21,341 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is no longer the leader 12:12:21,341 PM [cluster-test-shard-dispatcher-11] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 12:12:21,341 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 12:12:21,341 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 12:12:21,342 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 12:12:21,342 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:21,342 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 12:12:21,342 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 12:12:21,343 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 12:12:21,343 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 12:12:21,343 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 12:12:21,342 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 12:12:21,344 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 12:12:21,346 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 12:12:21,339 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 12:12:21,347 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 12:12:21,347 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 12:12:21,347 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 12:12:21,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:12:21,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:12:21,348 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 12:12:21,348 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 12:12:21,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-196544683, primaryVersion=13] 12:12:21,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-106203655, primaryVersion=13] 12:12:21,351 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 12:12:21,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1762564991, primaryVersion=13] 12:12:21,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 12:12:21,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 12:12:21,353 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 12:12:21,354 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#1676438525, primaryVersion=13] 12:12:21,354 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 12:12:21,354 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-2022719567, primaryVersion=13] 12:12:21,373 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-196544683 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,373 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-106203655 for shard member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,374 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1876823986] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,375 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#1811321570] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:21,378 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddReplicasForAllShards, persistent : true 12:12:21,378 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddReplicasForAllShards, persistent : true 12:12:21,378 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 12:12:21,379 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#220866781 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,380 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#1885408300] 12:12:21,382 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 12:12:21,382 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-699942628] 12:12:21,382 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#-2043153088 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,382 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 12:12:21,383 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1762564991 for shard member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,383 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#1676438525 for shard member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-2022719567 for shard member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,384 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testAddReplicasForAllShards, persistent : true 12:12:21,385 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#-121982620 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,385 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#1350111282] 12:12:21,385 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:21,385 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:21,385 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testAddReplicasForAllShards with journal batch size 1 12:12:21,386 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:12:21,386 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 12:12:21,386 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:21,386 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 12:12:21,386 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 12:12:21,386 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 12:12:21,386 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 12:12:21,386 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 12:12:21,387 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:21,387 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:21,387 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:21,387 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 12:12:21,387 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 12:12:21,387 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:12:21,387 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 12:12:21,387 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:12:21,388 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 12:12:21,388 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 12:12:21,389 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 12:12:21,389 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 12:12:21,389 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,389 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ClusterAdminRpcService - Failed to add replica for shard no-leader, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 12:12:21,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 12:12:21,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:21,391 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 12:12:21,391 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,391 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:21,391 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 12:12:21,391 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,391 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:21,389 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 12:12:21,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:21,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,392 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-people-oper_testAddReplicasForAllShards, persistent : true 12:12:21,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:21,393 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-807991840] 12:12:21,393 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#-9948281 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,394 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 12:12:21,394 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 12:12:21,395 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:21,395 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:21,396 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:12:21,396 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 12:12:21,396 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:12:21,396 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 12:12:21,396 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 12:12:21,396 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 12:12:21,396 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 12:12:21,398 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 12:12:21,398 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 12:12:21,398 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 12:12:21,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:21,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:21,399 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,404 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 12:12:21,404 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:21,398 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:21,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 12:12:21,405 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:21,405 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,405 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 12:12:21,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 12:12:21,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 12:12:21,405 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@55ceed90, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddReplicasForAllShards, ServerConfigPayload=null] 12:12:21,405 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:21,405 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@b2499ab, electionTerm=1, electionVotedFor=member-1-shard-pets-config_testAddReplicasForAllShards, ServerConfigPayload=null] 12:12:21,405 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@60431900, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 12:12:21,406 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:21,406 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:21,406 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,406 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 12:12:21,406 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@12155c27, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 12:12:21,406 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:21,406 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,407 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 12:12:21,407 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7e6e7273, electionTerm=1, electionVotedFor=member-1-shard-people-config_testAddReplicasForAllShards, ServerConfigPayload=null] 12:12:21,407 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:21,409 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 12:12:21,409 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 12:12:21,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-config_testAddReplicasForAllShards: 229 12:12:21,410 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddReplicasForAllShards: 229 12:12:21,411 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 12:12:21,411 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-pets-config_testAddReplicasForAllShards: 229 12:12:21,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 12:12:21,411 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 12:12:21,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-oper_testAddReplicasForAllShards: 229 12:12:21,411 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddReplicasForAllShards: 229 12:12:21,412 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-cars-oper_testAddReplicasForAllShards), Chunk: 1/1 12:12:21,412 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,412 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-pets-config_testAddReplicasForAllShards), Chunk: 1/1 12:12:21,413 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,413 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,413 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 12:12:21,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 12:12:21,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-cars-config_testAddReplicasForAllShards), Chunk: 1/1 12:12:21,416 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 12:12:21,418 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a journal response 12:12:21,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a journal response 12:12:21,418 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 12:12:21,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-people-oper_testAddReplicasForAllShards), Chunk: 1/1 12:12:21,418 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 12:12:21,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a journal response 12:12:21,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-people-config_testAddReplicasForAllShards), Chunk: 1/1 12:12:21,420 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 12:12:21,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a journal response 12:12:21,426 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:21,426 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:21,426 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,427 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:21,426 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:21,427 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,427 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:21,425 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:21,427 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,427 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:21,426 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,427 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:21,422 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:21,427 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,427 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:21,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:21,429 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-pets-config_testAddReplicasForAllShards 12:12:21,429 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:21,429 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-config_testAddReplicasForAllShards 12:12:21,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-pets-config_testAddReplicasForAllShards to state 12:12:21,430 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-people-config_testAddReplicasForAllShards to state 12:12:21,430 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:21,430 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddReplicasForAllShards 12:12:21,430 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddReplicasForAllShards to state 12:12:21,430 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:21,430 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-oper_testAddReplicasForAllShards 12:12:21,431 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-people-oper_testAddReplicasForAllShards to state 12:12:21,431 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 12:12:21,431 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 12:12:21,431 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:21,431 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:21,431 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddReplicasForAllShards to state 12:12:21,431 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:21,432 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 12:12:21,432 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 12:12:21,432 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 12:12:21,432 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3ddc925, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:21,432 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:21,432 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 12:12:21,432 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@19400d41, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:21,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 12:12:21,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,433 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,433 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 12:12:21,433 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 12:12:21,433 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 12:12:21,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:21,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:21,439 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-pets-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 12:12:21,439 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 12:12:21,439 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:21,439 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@307c2a17, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:21,440 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:21,440 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 12:12:21,441 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,441 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,441 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 12:12:21,441 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,441 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 12:12:21,441 PM [cluster-test-shard-dispatcher-35] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:21,439 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 12:12:21,443 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@55c28859, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:21,443 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:21,443 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 12:12:21,444 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,444 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,444 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 12:12:21,444 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,444 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 12:12:21,444 PM [cluster-test-shard-dispatcher-36] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:21,443 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3de0fe90, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:21,444 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:21,445 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 12:12:21,445 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:21,445 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 12:12:21,445 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 12:12:21,445 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 12:12:21,445 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 12:12:21,445 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:21,463 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#997759739] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 12:12:21,463 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#886586963] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 12:12:21,465 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#667451856] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 12:12:21,464 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 20.37 ms 12:12:21,465 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 24.07 ms 12:12:21,465 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 12:12:21,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 30.53 ms 12:12:21,465 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 12:12:21,466 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 12:12:21,466 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#886586963] 12:12:21,466 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#667451856] 12:12:21,466 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#997759739] 12:12:21,467 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 12:12:21,467 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 12:12:21,467 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 12:12:21,466 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,468 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,468 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a journal response 12:12:21,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1554655332] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 12:12:21,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 34.88 ms 12:12:21,469 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 12:12:21,469 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,469 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,469 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a journal response 12:12:21,469 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1554655332] 12:12:21,469 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#1110346792] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 12:12:21,469 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 12:12:21,470 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#1110346792] 12:12:21,470 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 12:12:21,469 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 24.39 ms 12:12:21,470 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 12:12:21,470 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,470 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,471 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a journal response 12:12:21,472 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 12:12:21,472 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 12:12:21,472 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 12:12:21,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 12:12:21,478 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 7.890 ms 12:12:21,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 10.55 ms 12:12:21,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a journal response 12:12:21,478 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 8.496 ms 12:12:21,478 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 10.72 ms 12:12:21,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,478 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:21,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:21,479 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-pets-config_testAddReplicasForAllShards] 12:12:21,479 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 11.44 ms 12:12:21,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,479 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:21,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:21,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-config_testAddReplicasForAllShards] 12:12:21,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,480 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,480 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:21,480 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:21,480 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:21,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:21,480 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-oper_testAddReplicasForAllShards] 12:12:21,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddReplicasForAllShards] 12:12:21,480 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]] 12:12:21,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]] 12:12:21,480 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]] 12:12:21,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]] 12:12:21,465 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,482 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 12:12:21,482 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 12:12:21,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 12:12:21,483 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:21,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:21,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddReplicasForAllShards] 12:12:21,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]] 12:12:21,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,483 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,483 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,486 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,486 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,486 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,483 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,488 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4c0a55c1-97fa-4b5b-9d19-36863d6529db}, logIndex: 0, isSendImmediate: true 12:12:21,489 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1ee2c406-bc6a-45a6-963c-6e720e4d11f3}, logIndex: 0, isSendImmediate: true 12:12:21,488 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=6f09558f-021c-44f6-883d-93eb921f1400}, logIndex: 0, isSendImmediate: true 12:12:21,488 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=2c7333f5-84db-43ee-b549-2c5f5743382c}, logIndex: 0, isSendImmediate: true 12:12:21,489 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=432b5846-3cb9-456c-a1d3-53ada61b117d}, logIndex: 0, isSendImmediate: true 12:12:21,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,490 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,490 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 12:12:21,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,490 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,490 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,499 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,499 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,502 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,502 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,506 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 12:12:21,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,516 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1762564991 12:12:21,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 12:12:21,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:21,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,517 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 12:12:21,517 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:21,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,517 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,517 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testAddReplicasForAllShards, address=null, votingState=VOTING]] 12:12:21,518 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Log size is now 1 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards, votingState=VOTING]] 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Log size is now 1 12:12:21,516 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-106203655 12:12:21,519 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 12:12:21,519 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,519 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 12:12:21,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards, votingState=VOTING]] 12:12:21,516 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 12:12:21,516 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 12:12:21,520 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,520 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Log size is now 1 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards, votingState=VOTING]] 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Log size is now 1 12:12:21,520 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,520 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [people] 12:12:21,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:12:21,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 12:12:21,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:12:21,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 12:12:21,522 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:21,518 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:21,520 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:21,520 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:21,522 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-pets-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 12:12:21,522 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 12:12:21,522 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 12:12:21,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:21,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 12:12:21,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,523 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:21,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:21,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddReplicasForAllShards 12:12:21,524 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 12:12:21,524 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 12:12:21,523 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 12:12:21,525 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,526 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:21,526 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-oper_testAddReplicasForAllShards 12:12:21,526 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 12:12:21,526 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 12:12:21,526 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 12:12:21,526 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 12:12:21,526 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:21,526 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:21,526 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:21,526 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:21,526 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:21,527 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 12:12:21,527 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 12:12:21,527 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 12:12:21,527 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 12:12:21,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#1676438525 12:12:21,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards, votingState=VOTING]] 12:12:21,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars, people] 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Log size is now 1 12:12:21,528 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:21,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-2022719567 12:12:21,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:21,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:21,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:21,529 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-196544683 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-config_testAddReplicasForAllShards 12:12:21,529 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:21,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-pets-config_testAddReplicasForAllShards 12:12:21,529 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 12:12:21,530 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 12:12:21,530 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,530 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 12:12:21,530 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 12:12:21,530 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 12:12:21,530 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:21,530 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 12:12:21,530 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:21,530 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 12:12:21,530 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 12:12:21,530 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,530 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,530 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:12:21,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,531 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 12:12:21,531 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:21,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,531 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 12:12:21,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 12:12:21,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:21,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 12:12:21,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,532 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:21,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:21,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,531 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 12:12:21,531 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 12:12:21,534 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 12:12:21,534 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 12:12:21,534 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:12:21,534 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:21,534 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:21,534 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 12:12:21,534 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 12:12:21,534 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 12:12:21,531 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:21,534 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 12:12:21,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 12:12:21,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,535 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,535 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:21,535 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,536 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#367713322] 12:12:21,536 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:21,537 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Commit index set to 0 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,538 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,538 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 12:12:21,539 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:21,539 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 12:12:21,539 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,539 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,539 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#1811321570] 12:12:21,539 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#1811321570] 12:12:21,539 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Commit index set to 0 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,540 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 12:12:21,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#367713322] 12:12:21,541 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:12:21,541 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#994686328] 12:12:21,541 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#994686328] 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Commit index set to 0 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,542 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 12:12:21,542 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:21,542 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 12:12:21,542 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 12:12:21,542 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 12:12:21,542 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#367713322] 12:12:21,542 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 12:12:21,543 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 12:12:21,543 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 12:12:21,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#909739147] 12:12:21,543 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 12:12:21,543 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#909739147] 12:12:21,543 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1788169108] 12:12:21,543 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:12:21,543 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1876823986] 12:12:21,543 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1876823986] 12:12:21,543 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1788169108] 12:12:21,544 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 12:12:21,544 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#669307512] 12:12:21,544 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#669307512] 12:12:21,544 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 12:12:21,544 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,544 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,545 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onShutDown 12:12:21,545 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onShutDown 12:12:21,545 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testAddReplicasForAllShards 12:12:21,545 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddReplicasForAllShards 12:12:21,546 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:21,546 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:21,546 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,546 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 12:12:21,546 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onShutDown 12:12:21,547 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-2-shard-people-config_testAddReplicasForAllShards 12:12:21,547 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:21,547 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:12:21,547 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:12:21,548 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 12:12:21,549 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,549 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,549 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 12:12:21,549 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 12:12:21,549 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onShutDown 12:12:21,550 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onShutDown 12:12:21,550 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onShutDown 12:12:21,550 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddReplicasForAllShards 12:12:21,550 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:21,550 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testAddReplicasForAllShards 12:12:21,550 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:21,551 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 12:12:21,551 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:21,551 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:12:21,551 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:12:21,554 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:21,554 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:21,580 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:21,580 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:21,589 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:12:21,612 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 12:12:21,612 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 12:12:21,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 12:12:21,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 12:12:21,613 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 12:12:21,613 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onShutDown 12:12:21,613 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onShutDown 12:12:21,613 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onShutDown 12:12:21,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating leader transfer 12:12:21,614 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating leader transfer 12:12:21,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating leader transfer 12:12:21,617 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 12:12:21,617 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 12:12:21,617 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 12:12:21,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@477960ac 12:12:21,618 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1491f9b8 12:12:21,618 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7cfb1b2d 12:12:21,618 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1491f9b8 12:12:21,619 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7cfb1b2d 12:12:21,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 12:12:21,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 12:12:21,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@477960ac 12:12:21,619 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 12:12:21,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 12:12:21,620 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 12:12:21,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 12:12:21,621 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:21,621 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:22,579 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3444717659799583648] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:12:25,873 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:26,191 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:26,341 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:27,152 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:27,334 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 12:12:27,336 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:27,337 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:27,362 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:27,789 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:28,319 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:28,381 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:28,754 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:29,71 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:29,398 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:29,398 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:29,710 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:30,31 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:30,330 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:30,420 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:30,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 12:12:30,658 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 12:12:30,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 12:12:30,658 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.042 s 12:12:30,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 12:12:30,658 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.042 s 12:12:30,659 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 12:12:30,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 12:12:30,659 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.042 s 12:12:30,659 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 12:12:30,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 12:12:30,659 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 12:12:30,659 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 12:12:30,659 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 12:12:30,659 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 12:12:30,660 PM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,660 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testAddReplicasForAllShards] 12:12:30,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,662 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:12:30,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:30,662 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 12:12:30,662 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,662 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-pets-config_testAddReplicasForAllShards] 12:12:30,663 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,663 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-config_testAddReplicasForAllShards 12:12:30,663 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,663 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:30,663 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:12:30,663 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:30,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 12:12:30,664 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,664 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testAddReplicasForAllShards] 12:12:30,664 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,664 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testAddReplicasForAllShards 12:12:30,664 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:30,664 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:12:30,664 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:30,665 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 12:12:30,665 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,666 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddReplicasForAllShards 12:12:30,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:30,667 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:12:30,667 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:12:30,667 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 12:12:30,668 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:30,668 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 12:12:30,668 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:12:30,668 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onShutDown 12:12:30,668 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating leader transfer 12:12:30,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onShutDown 12:12:30,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating leader transfer 12:12:30,669 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ad3d738 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ad3d738 12:12:30,669 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testAddReplicasForAllShards] 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: leader transfer aborted 12:12:30,669 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: Failed to transfer leadership in 836.0 μs 12:12:30,670 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: In unpauseLeader 12:12:30,670 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 12:12:30,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@301f1ba6 12:12:30,670 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:30,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@301f1ba6 12:12:30,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 12:12:30,670 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 12:12:30,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 12:12:30,670 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,670 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testAddReplicasForAllShards 12:12:30,670 PM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,670 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:30,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testAddReplicasForAllShards] 12:12:30,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:30,671 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:12:30,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddReplicasForAllShards: leader transfer aborted 12:12:30,672 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddReplicasForAllShards: Failed to transfer leadership in 2.967 ms 12:12:30,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: In unpauseLeader 12:12:30,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 12:12:30,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:12:30,672 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 12:12:30,672 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 12:12:30,672 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddReplicasForAllShards 12:12:30,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 12:12:30,672 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:12:30,672 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:12:30,674 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:30,674 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:30,674 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:30,679 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:12:30.684] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:30,711 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:30,734 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3513478583581614292] 12:12:30,735 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:12:30,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:12:30,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:12:30,740 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:30,740 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:30,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:12:30,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:12:30,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:12:30,777 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,780 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,783 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 12:12:30,789 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,791 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:12:30,799 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:12:30,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 12:12:30,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 12:12:30,801 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:30,801 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:30,801 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:30,807 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:30,809 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5326a4c 12:12:30,809 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 12:12:30,810 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:30,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 12:12:30,815 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,817 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:30,817 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:30,817 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:30,817 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 12:12:30,818 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:30,818 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:30,818 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#905391769] 12:12:30,818 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 12:12:30,819 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStates, persistent : true 12:12:30,819 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,821 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#308169150] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,823 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#1465723770] 12:12:30,823 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:12:30,823 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStates, persistent : true 12:12:30,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#1317963942 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 12:12:30,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#1465723770] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1879838286 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 12:12:30,826 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 12:12:30,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,836 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#905391769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,836 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#1317963942] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,836 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1879838286] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 12:12:30,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:30,837 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,837 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,837 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:12:30,837 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 12:12:30,837 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,837 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:30,837 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,838 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#392974406] 12:12:30,839 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStates with journal batch size 1 12:12:30,839 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#392974406] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,839 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 12:12:30,839 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 12:12:30,842 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,842 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,842 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:12:30,842 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:30,842 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 12:12:30,843 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:30,843 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:30,843 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:30,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 12:12:30,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 12:12:30,844 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,844 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:12:30,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 12:12:30,844 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,844 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:30,844 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,844 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 12:12:30,845 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 12:12:30,846 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:30,847 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2aa88a6a 12:12:30,847 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 12:12:30,848 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:30,848 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:30,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#422791296] 12:12:30,850 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStates, persistent : true 12:12:30,851 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 12:12:30,851 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStates, persistent : true 12:12:30,851 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-599482876 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:30,851 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,851 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#138745792] 12:12:30,851 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#422791296] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,852 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#-110422733 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 12:12:30,852 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#1681747862] 12:12:30,852 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 12:12:30,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,855 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,855 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,855 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 12:12:30,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 12:12:30,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 12:12:30,856 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:30,856 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:30,856 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:12:30,856 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 12:12:30,856 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 12:12:30,856 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 12:12:30,857 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 12:12:30,857 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 12:12:30,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:30,878 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [50758897619016917] 12:12:30,879 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:12:30,882 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:30,882 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:12:30,883 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:30,883 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:30,910 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,913 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,916 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 12:12:30,924 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1075936609] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,925 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:12:30,933 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:12:30,933 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 12:12:30,934 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 12:12:30,938 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:30,940 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@a2e1570 12:12:30,940 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 12:12:30,942 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,944 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:30,946 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:30,946 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 12:12:30,947 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 12:12:30,947 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStates, persistent : true 12:12:30,947 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-435385969] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,948 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#800241131] 12:12:30,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#324599421 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 12:12:30,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#324599421] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,949 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 12:12:30,949 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#800241131] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,949 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStates, persistent : true 12:12:30,949 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#118785470] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,950 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStates with journal batch size 1 12:12:30,951 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-31050463 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 12:12:30,951 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-31050463] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,951 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-1159696718] 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 12:12:30,951 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-1159696718] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,951 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,951 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:30,951 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,953 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:12:30,955 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:12:30,955 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 12:12:30,955 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 12:12:30,955 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 12:12:30,963 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,963 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,963 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,963 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 12:12:30,964 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,964 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:30,964 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:12:30,964 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 12:12:30,964 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,964 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 12:12:30,964 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 12:12:30,965 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-185864253] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,969 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:30,970 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b3ddcea 12:12:30,970 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 12:12:30,971 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:30,971 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:30,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 12:12:30,973 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStates, persistent : true 12:12:30,973 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1038142878] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,974 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#937089442 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:30,974 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#1673082448] 12:12:30,974 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#937089442] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:30,974 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 12:12:30,974 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStates, persistent : true 12:12:30,975 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 12:12:30,976 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#2066748485 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 12:12:30,976 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#-2144400666] 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,976 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,977 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,977 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:30,977 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:12:30,977 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 12:12:30,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 12:12:30,977 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:30,978 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:30,978 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:30,978 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 12:12:30,978 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 12:12:30,978 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 12:12:30,993 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:30,995 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,2 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:31,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7089854714130311374] 12:12:31,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:12:31,28 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:31,28 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:12:31,40 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:31,40 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:31,45 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:31,47 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:31,51 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 12:12:31,76 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1873231215] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,84 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:12:31,84 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:12:31,88 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:31,95 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:12:31,95 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,95 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,96 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:31,96 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,96 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,97 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:31,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4597609d 12:12:31,98 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 12:12:31,100 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:31,102 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:31,103 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:31,103 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-3-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1888271724] 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,104 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-3-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1186992516] 12:12:31,105 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStates, persistent : true 12:12:31,105 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1888271724] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,106 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#-812352675 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,106 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1777827711] 12:12:31,106 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#-812352675] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,109 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 12:12:31,111 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:12:31,111 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStates, persistent : true 12:12:31,112 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#549020324] 12:12:31,112 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#338809017 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,112 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStates with journal batch size 1 12:12:31,114 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:31,114 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:31,114 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:31,115 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1777827711] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,115 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1533133202] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,115 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1186992516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,115 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#338809017] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,115 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#549020324] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,116 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:31,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:12:31,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:31,117 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,117 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 12:12:31,117 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 12:12:31,117 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 12:12:31,119 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:12:31,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,120 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:31,120 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,120 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:31,120 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:12:31,120 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:31,120 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 12:12:31,120 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,120 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 12:12:31,121 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 12:12:31,122 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:31,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4e54ec70 12:12:31,122 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 12:12:31,123 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:31,123 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:31,123 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,124 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,124 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,124 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-3-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1571091299] 12:12:31,124 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,124 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,125 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,125 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-3-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#820396065] 12:12:31,125 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStates, persistent : true 12:12:31,125 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1571091299] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,126 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1142773319] 12:12:31,126 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#747301153 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,126 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 12:12:31,126 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#747301153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:31,126 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStates, persistent : true 12:12:31,127 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-1933524808 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,127 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1335153288] 12:12:31,127 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 12:12:31,129 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:31,129 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:12:31,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 12:12:31,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:12:31,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 12:12:31,130 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 12:12:31,130 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 12:12:31,135 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 12:12:31,135 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,135 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:31,135 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 12:12:31,135 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,135 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 12:12:31,135 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 12:12:31,755 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:12:31,756 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:12:31,756 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:31,756 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:31,756 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,756 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,756 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,756 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,757 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,775 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:31,776 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,776 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:31,777 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:31,777 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,777 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:31,884 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:31,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 12:12:31,884 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 12:12:31,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 12:12:31,885 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 12:12:31,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,885 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,885 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 12:12:31,885 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 12:12:31,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,885 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,889 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:12:31,889 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:12:31,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,889 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:31,889 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:31,889 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:31,889 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:31,890 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 12:12:31,889 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:12:31,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,897 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:12:31,898 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:12:31,909 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:12:31,909 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:12:31,909 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:12:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 12:12:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:31,912 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,912 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,912 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:12:31,912 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:12:31,912 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:12:31,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,912 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 12:12:31,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,913 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,913 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,913 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 12:12:31,913 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 12:12:31,914 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,914 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,914 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 12:12:31,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,915 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,918 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 12:12:31,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,918 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,919 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 12:12:31,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 12:12:31,921 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,921 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,922 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 12:12:31,922 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,923 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,924 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:31,925 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 12:12:31,925 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:31,925 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:31,926 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 12:12:31,924 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:31,926 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,926 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,926 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,926 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,926 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,926 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,927 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,927 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 12:12:31,928 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 12:12:31,928 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,928 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 12:12:31,929 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,929 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-2-shard-people-oper_testFlipMemberVotingStates] 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,933 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 12:12:31,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,934 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,934 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,934 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,934 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:12:31,934 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:12:31,934 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,934 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:31,934 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 12:12:31,934 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,935 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:12:31,935 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:31,935 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,935 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,936 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 12:12:31,936 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:12:31,936 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 12:12:31,936 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,936 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,936 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:12:31,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,943 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,944 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 12:12:31,944 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,944 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,944 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:31,944 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,944 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,944 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 12:12:31,944 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 12:12:31,944 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,945 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,945 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 2 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 12:12:31,945 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,945 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:12:31,945 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,945 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 12:12:31,945 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,945 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,945 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,946 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 12:12:31,946 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,947 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,947 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,947 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,947 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,947 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,947 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:12:31,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:12:31,948 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:12:31,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 12:12:31,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,950 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,950 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 12:12:31,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,951 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,951 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,951 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,952 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,952 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 12:12:31,952 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 12:12:31,953 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,953 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,953 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:31,953 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 12:12:31,954 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 2 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,954 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,955 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,955 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:31,955 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:31,955 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 12:12:31,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,955 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 12:12:31,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,956 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,956 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,956 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,957 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,958 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 12:12:31,958 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 12:12:31,958 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:31,958 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 12:12:31,958 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,959 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,959 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,960 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:31,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,961 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,969 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 12:12:31,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,972 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:31,972 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:12:31,972 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:31,972 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 12:12:31,972 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 12:12:31,972 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:12:31,972 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:31,972 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 12:12:31,972 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:31,972 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 12:12:31,972 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:31,972 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 12:12:31,973 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:31,973 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:31,973 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:31,973 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:31,973 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,974 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:31,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 12:12:31,981 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 12:12:31,981 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 12:12:31,981 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 12:12:31,982 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:12:31,983 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:12:31,984 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:12:31,984 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:12:31,984 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:12:31,984 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:12:31,984 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:31,984 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:12:31,986 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:31,986 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:31,986 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:31,986 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 12:12:31,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 12:12:31,987 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 12:12:31,987 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:31,987 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:31,987 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates)] 12:12:31,987 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates)] 12:12:31,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:31,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates)] 12:12:31,987 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:31,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 12:12:31,988 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:31,988 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates)] 12:12:31,998 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 12:12:31,998 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 12:12:31,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 12:12:31,999 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 12:12:32,1 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,1 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d52e71c5-6a9e-4dc2-89e8-c428ebe36b1e}, logIndex: 2, isSendImmediate: true 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,2 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,3 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 12:12:32,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,3 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 12:12:32,3 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,3 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=ce73ad19-de38-4bf4-95cd-b2ff6dc64fb8}, logIndex: 2, isSendImmediate: true 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f52dc026-5e98-4443-b52d-68405b8a86ab}, logIndex: 2, isSendImmediate: true 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 12:12:32,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] 12:12:32,6 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,7 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,8 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,8 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,8 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,8 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,8 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,8 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,9 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,9 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,9 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,9 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,9 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,10 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,10 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,10 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,10 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 3 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 3 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:32,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:12:32,11 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 12:12:32,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,11 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 3 12:12:32,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:32,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,12 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,13 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,13 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 12:12:32,13 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:12:32,13 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 12:12:32,13 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,13 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,13 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,13 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 12:12:32,14 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,14 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,14 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65321cd3 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@728a1e44 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65321cd3 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@728a1e44 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,15 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,16 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,16 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,16 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,17 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,17 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,17 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,17 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@149ac7a8 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@149ac7a8 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,18 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,19 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,19 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 12:12:32,21 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,21 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,21 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4d267983-aaba-49df-8fc0-36469700ac54}, logIndex: 2, isSendImmediate: true 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 12:12:32,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] 12:12:32,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 2 12:12:32,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 1 12:12:32,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,23 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,23 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 12:12:32,23 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,23 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,23 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,22 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,24 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,24 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,24 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,24 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,24 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,24 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,24 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 12:12:32,24 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 12:12:32,24 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,25 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,26 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,26 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 12:12:32,26 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,26 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 3 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 12:12:32,27 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:32,28 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:32,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,28 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,29 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 12:12:32,30 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 12:12:32,30 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 12:12:32,30 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 12:12:32,30 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,30 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 12:12:32,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,31 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,31 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,31 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,31 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 12:12:32,31 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,31 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,32 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,32 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,32 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,32 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:32,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:12:32,32 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#905391769] 12:12:32,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#905391769] 12:12:32,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#308169150] 12:12:32,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:12:32,33 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:32,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:32,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#308169150] 12:12:32,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:12:32,33 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#422791296] 12:12:32,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#422791296] 12:12:32,34 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1075936609] 12:12:32,34 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:32,34 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-435385969] 12:12:32,34 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-435385969] 12:12:32,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,34 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1075936609] 12:12:32,35 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:12:32,35 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#118785470] 12:12:32,35 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#118785470] 12:12:32,35 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-185864253] 12:12:32,35 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:12:32,35 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1038142878] 12:12:32,35 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1038142878] 12:12:32,35 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-185864253] 12:12:32,35 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 12:12:32,36 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#212807069] 12:12:32,36 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#212807069] 12:12:32,36 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:12:32,36 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1873231215] 12:12:32,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:32,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1888271724] 12:12:32,36 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1888271724] 12:12:32,36 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,36 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 12:12:32,36 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1873231215] 12:12:32,37 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:12:32,37 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 12:12:32,37 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1186992516] 12:12:32,37 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1186992516] 12:12:32,37 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,37 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1533133202] 12:12:32,37 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:12:32,37 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,37 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1571091299] 12:12:32,37 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1571091299] 12:12:32,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,38 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1533133202] 12:12:32,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:12:32,38 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#820396065] 12:12:32,38 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#820396065] 12:12:32,38 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:32,38 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:32,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,38 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,39 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,39 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,39 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@20ab7e5d 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@20ab7e5d 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,39 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,39 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 12:12:32,39 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,39 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 12:12:32,39 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,40 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,40 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:32,40 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 12:12:32,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 12:12:32,40 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-1-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 12:12:32,40 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,41 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,41 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:32,41 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,41 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,41 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,41 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,41 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,41 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-2-shard-people-config_testFlipMemberVotingStates] 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 12:12:32,41 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,41 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,41 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,42 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:32,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,42 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,42 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,42 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 12:12:32,42 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,43 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 12:12:32,43 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 12:12:32,43 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,43 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,43 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,44 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,44 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,44 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,44 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,44 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 12:12:32,44 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,46 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 12:12:32,46 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,46 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 12:12:32,46 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,46 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 12:12:32,46 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 12:12:32,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,46 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 12:12:32,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:32,47 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 12:12:32,47 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,47 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 12:12:32,47 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,47 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,47 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 12:12:32,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,48 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,48 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 12:12:32,48 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 12:12:32,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 12:12:32,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 12:12:32,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 12:12:32,48 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,48 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,49 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,49 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 12:12:32,50 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 12:12:32,50 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 12:12:32,50 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,50 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,50 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,50 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,50 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,50 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,50 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,50 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,51 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:12:32,51 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,51 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 12:12:32,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,52 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:32,53 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 12:12:32,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,54 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 12:12:32,54 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,55 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 12:12:32,55 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 12:12:32,57 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 12:12:32,57 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,57 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 12:12:32,57 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,57 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:32,57 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 12:12:32,57 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 12:12:32,57 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 12:12:32,58 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,58 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,58 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,58 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 12:12:32,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,62 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 12:12:32,93 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,194 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,245 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,296 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,347 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,366 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,367 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,367 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 351.9 ms 12:12:32,367 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 12:12:32,367 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 12:12:32,367 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,368 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,368 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,368 PM [cluster-test-shard-dispatcher-16] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 352.8 ms 12:12:32,368 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 12:12:32,368 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 12:12:32,368 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,368 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,368 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,368 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 351.0 ms 12:12:32,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 12:12:32,369 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 12:12:32,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,370 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,371 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 12:12:32,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,375 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,375 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 336.2 ms 12:12:32,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 12:12:32,375 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,375 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 12:12:32,375 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 12:12:32,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 12:12:32,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 12:12:32,379 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:12:32,381 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 12:12:32,398 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,399 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#308169150] 12:12:32,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:12:32,399 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:32,399 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:32,400 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 12:12:32,400 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 12:12:32,400 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:12:32,400 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 12:12:32,400 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:12:32,401 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:12:32,401 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:12:32,401 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:32,401 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:12:32,401 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:12:32,401 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:12:32,402 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,402 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,402 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,403 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8a648840-e1c7-4c44-8c38-3cea29ca8d10}, logIndex: 3, isSendImmediate: true 12:12:32,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 12:12:32,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,403 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,404 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=aef08c24-440a-4664-aea9-ddbb26a26747}, logIndex: 3, isSendImmediate: true 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] 12:12:32,404 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,404 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 12:12:32,404 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=81fdcfef-f132-4f98-9acb-4f604a38cda9}, logIndex: 3, isSendImmediate: true 12:12:32,404 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,404 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 12:12:32,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 12:12:32,405 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,405 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,405 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 12:12:32,405 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,405 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:12:32,405 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,405 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=0924665b-dba1-45db-b906-e848d47b65a9}, logIndex: 3, isSendImmediate: true 12:12:32,406 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,406 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,406 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,407 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,407 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:32,407 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,407 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 12:12:32,407 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,407 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 12:12:32,407 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:12:32,407 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 12:12:32,408 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,408 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:12:32,408 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 12:12:32,409 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:32,409 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:32,409 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,409 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,411 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,411 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,411 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 12:12:32,414 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@334f840 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@334f840 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6044e6c9 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6044e6c9 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 12:12:32,415 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7deda2b 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7deda2b 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@46366642 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@46366642 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,417 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,417 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,417 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,416 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,419 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,419 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,424 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,425 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-oper_testFlipMemberVotingStates] 12:12:32,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,428 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 12:12:32,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-people-oper_testFlipMemberVotingStates 12:12:32,428 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 12:12:32,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,428 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,428 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,428 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:12:32,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 12:12:32,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,432 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,432 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 12:12:32,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,434 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,434 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,434 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,435 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,435 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,435 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,435 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 12:12:32,436 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,437 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,437 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,439 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,439 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,440 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 12:12:32,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 12:12:32,441 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-config_testFlipMemberVotingStates 12:12:32,441 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:32,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,441 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,441 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 12:12:32,441 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,441 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,441 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-config_testFlipMemberVotingStates 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:32,442 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 12:12:32,442 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,442 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 12:12:32,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:12:32,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:12:32,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,444 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:12:32,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 12:12:32,444 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,446 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,446 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 12:12:32,447 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,447 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,449 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 12:12:32,450 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 12:12:32,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,450 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 12:12:32,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,450 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,451 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,451 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 12:12:32,455 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:12:32,455 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:12:32,455 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:12:32,455 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:12:32,455 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 12:12:32,455 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 12:12:32,455 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,455 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,455 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,455 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,456 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 12:12:32,456 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,456 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,456 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,456 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 12:12:32,456 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,458 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 12:12:32,458 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,458 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 12:12:32,458 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 12:12:32,458 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,458 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 12:12:32,458 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 12:12:32,458 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 12:12:32,459 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,459 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 12:12:32,459 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,459 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 12:12:32,459 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,459 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,459 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 12:12:32,459 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,459 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 12:12:32,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:32,461 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:12:32,461 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,461 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,461 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,461 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#905391769] 12:12:32,461 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#905391769] 12:12:32,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#308169150] 12:12:32,462 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:12:32,462 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:32,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1573572762] 12:12:32,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#308169150] 12:12:32,462 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:12:32,463 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#422791296] 12:12:32,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#422791296] 12:12:32,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1075936609] 12:12:32,463 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:32,463 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-435385969] 12:12:32,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-435385969] 12:12:32,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1075936609] 12:12:32,463 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:12:32,464 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#118785470] 12:12:32,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#118785470] 12:12:32,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-185864253] 12:12:32,464 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:12:32,464 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 12:12:32,464 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1038142878] 12:12:32,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1038142878] 12:12:32,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-185864253] 12:12:32,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 12:12:32,465 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#212807069] 12:12:32,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#212807069] 12:12:32,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1873231215] 12:12:32,465 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:32,466 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1888271724] 12:12:32,466 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1888271724] 12:12:32,466 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 12:12:32,466 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1873231215] 12:12:32,466 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:12:32,466 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1186992516] 12:12:32,466 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-1186992516] 12:12:32,467 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1533133202] 12:12:32,467 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:12:32,467 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1571091299] 12:12:32,467 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1571091299] 12:12:32,467 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1533133202] 12:12:32,467 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:12:32,467 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#820396065] 12:12:32,467 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#820396065] 12:12:32,467 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#830652674] 12:12:32,468 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:32,468 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,468 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-94992619] 12:12:32,468 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:32,468 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,468 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,468 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 12:12:32,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onShutDown 12:12:32,469 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onShutDown 12:12:32,469 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStates 12:12:32,469 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStates 12:12:32,469 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,469 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,469 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:12:32,470 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:12:32,470 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:32,470 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,470 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,470 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:12:32,470 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onShutDown 12:12:32,470 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onShutDown 12:12:32,471 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStates 12:12:32,471 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,471 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStates 12:12:32,471 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,471 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:12:32,471 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:12:32,472 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:32,472 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:32,479 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:32,480 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:32,481 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:32,482 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:32,485 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:12:32,512 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:32,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 12:12:32,514 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 12:12:32,514 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:12:32,514 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onShutDown 12:12:32,514 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onShutDown 12:12:32,514 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStates 12:12:32,514 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,515 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStates 12:12:32,515 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,515 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:12:32,515 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:12:32,516 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:32,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,516 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:12:32,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onShutDown 12:12:32,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f56993b 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f56993b 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onShutDown 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@40b07e8e 12:12:32,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@40b07e8e 12:12:32,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:32,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:32,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,518 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,518 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,522 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,522 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,522 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-people-oper_testFlipMemberVotingStates] 12:12:32,522 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,524 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,525 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 12:12:32,525 PM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 12:12:32,526 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,527 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 12:12:32,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,527 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 12:12:32,527 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 12:12:32,527 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 12:12:32,527 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,528 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,528 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,528 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 12:12:32,529 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:32,529 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Cluster state: [] 12:12:32,529 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 12:12:32,529 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Cluster state: [] 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 12:12:32,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 12:12:32,531 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,531 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,531 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 12:12:32,531 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 12:12:32,531 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 12:12:32,531 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 12:12:32,531 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 12:12:32,531 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 12:12:32,531 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 12:12:32,532 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,532 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 12:12:32,532 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 12:12:32,532 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 12:12:32,533 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 12:12:32,533 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 12:12:32,533 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 17.30 ms 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader changed to member-1-shard-people-oper_testFlipMemberVotingStates 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-oper_testFlipMemberVotingStates in 17.04 ms 12:12:32,534 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 12:12:32,535 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 12:12:32,535 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStates 12:12:32,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,535 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStates 12:12:32,535 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:32,535 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:12:32,536 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:12:32,536 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,536 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:32,537 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 12:12:32,537 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:32,537 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:32,537 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:32,774 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:12:33,494 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-7089854714130311374] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:12:33,554 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [50758897619016917] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:12:33,562 PM [cluster-test-akka.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:33,567 PM [cluster-test-akka.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:33,569 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:12:33.572] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:33,588 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 12:12:33,590 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 12:12:33,590 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 12:12:33,591 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onShutDown 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 12:12:33,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onShutDown 12:12:33,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 12:12:33,591 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:33,591 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7008683e 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7008683e 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:33,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@223fcdf6 12:12:33,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@223fcdf6 12:12:33,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:33,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:33,591 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:33,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:33,592 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:33,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:34,613 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 12:12:34,614 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 12:12:34,614 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.023 s 12:12:34,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 12:12:34,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 12:12:34,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 12:12:34,614 PM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.023 s 12:12:34,614 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 12:12:34,614 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 12:12:34,614 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 12:12:34,615 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStates 12:12:34,615 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStates 12:12:34,615 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:34,616 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:34,616 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:12:34,617 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:12:34,617 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 12:12:34,618 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:34,618 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 12:12:34,618 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:12:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onShutDown 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onShutDown 12:12:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 12:12:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1930091 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69f02839 12:12:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1930091 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69f02839 12:12:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 12:12:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:34,619 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:34,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 12:12:34,619 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 12:12:34,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:34,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:12:35,643 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 12:12:35,643 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 12:12:35,644 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.025 s 12:12:35,644 PM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.025 s 12:12:35,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 12:12:35,644 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 12:12:35,644 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 12:12:35,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 12:12:35,644 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 12:12:35,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 12:12:35,645 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStates 12:12:35,646 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStates 12:12:35,646 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:35,646 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 12:12:35,646 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:12:35,647 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:12:35,650 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:35,650 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:35,655 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:12:35.659] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:35,675 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:12:35,689 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1536512330992331036] 12:12:35,690 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:12:35,693 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:12:35,693 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:12:35,696 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:35,696 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:35,712 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:12:35,713 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:12:35,713 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:12:35,715 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,717 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,719 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 12:12:35,727 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2062831336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,729 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:12:35,737 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:12:35,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,739 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:35,739 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,739 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 12:12:35,745 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7e735489 12:12:35,745 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 12:12:35,746 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:35,746 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,748 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testModuleShardLeaderMovement, persistent : true 12:12:35,748 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,749 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-1951394775 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,749 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-1834091264] 12:12:35,749 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-1951394775] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,749 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:35,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,751 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-1834091264] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,755 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,755 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,755 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-567542449] 12:12:35,756 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testModuleShardLeaderMovement, persistent : true 12:12:35,756 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-567542449] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,757 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testModuleShardLeaderMovement with journal batch size 1 12:12:35,757 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-362309472 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,757 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1382387486] 12:12:35,757 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-362309472] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,758 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1382387486] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,758 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:35,759 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,759 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 12:12:35,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-config_testModuleShardLeaderMovement 12:12:35,760 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 12:12:35,760 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 12:12:35,760 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:35,760 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,760 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 12:12:35,760 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:35,761 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:35,761 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:35,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 12:12:35,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:12:35,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 12:12:35,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 12:12:35,761 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:35,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:35,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:35,761 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:35,761 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 12:12:35,763 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:35,763 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:35,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 12:12:35,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 12:12:35,768 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:35,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:35,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:35,768 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 12:12:35,768 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:35,768 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:35,768 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 12:12:35,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:35,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 12:12:35,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:35,796 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2062831336] 12:12:35,797 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:35,797 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:35,797 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,801 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,807 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 12:12:35,809 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-563188089] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,810 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:12:35,812 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:12:35,812 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,812 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,812 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:35,812 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,812 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,816 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 12:12:35,817 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@254ce347 12:12:35,817 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 12:12:35,818 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:35,818 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020] 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,819 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 12:12:35,819 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,819 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#-1888927187] 12:12:35,820 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-2012474694] 12:12:35,820 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testModuleShardLeaderMovement, persistent : true 12:12:35,820 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-755509042 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,820 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 12:12:35,820 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-755509042] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,820 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1472307768] 12:12:35,820 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-234076541 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,821 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testModuleShardLeaderMovement with journal batch size 1 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:35,822 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:12:35,822 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:35,822 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:35,822 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:35,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 12:12:35,822 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,822 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:35,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:35,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 12:12:35,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:35,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 12:12:35,823 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 12:12:35,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 12:12:35,823 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 12:12:35,823 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:35,823 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:12:35,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:35,823 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:35,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:35,823 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:35,823 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:35,823 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:35,823 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 12:12:35,823 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 12:12:35,824 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:35,824 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:35,824 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 12:12:35,824 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 12:12:35,824 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:35,868 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-563188089] 12:12:35,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:12:35,869 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020] 12:12:35,884 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:35,898 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5724800490223156652] 12:12:35,899 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:12:35,903 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:35,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:12:35,904 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:35,904 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:35,929 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,931 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,936 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 12:12:35,947 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 12:12:35,948 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 12:12:35,949 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,951 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:35,954 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 12:12:35,961 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:12:35,961 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1459788360] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,961 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:12:35,962 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-908225933] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:35,966 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 12:12:35,967 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:12:35,967 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:12:35,967 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4a911d06 12:12:35,967 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:35,969 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@40697676 12:12:35,969 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 12:12:35,970 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:36,8 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:12:36,8 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:36,9 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:36,12 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:36,12 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:36,12 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,12 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,27 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7605167928901664707] 12:12:36,28 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:12:36,32 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:36,32 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:12:36,44 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:36,44 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:36,61 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:36,62 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:36,66 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 12:12:36,76 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:36,76 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:12:36,81 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:12:36,83 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 12:12:36,84 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2b6e6239 12:12:36,84 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 12:12:36,86 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:36,86 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:36,89 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:36,102 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 12:12:36,104 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:12:36,105 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1278494415] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:36,105 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:12:36,107 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:12:36,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:36,111 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 12:12:36,112 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:36,112 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 12:12:36,112 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,112 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:36,112 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@21960949 12:12:36,113 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:36,713 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:12:36,713 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 12:12:36,714 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,726 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:36,726 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:36,726 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,727 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,727 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:36,727 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:36,727 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,727 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:36,940 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:37,18 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:12:37,19 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 12:12:37,20 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,20 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 12:12:37,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702, primaryVersion=13] 12:12:37,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,29 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-cars-config_testModuleShardLeaderMovement, persistent : true 12:12:37,29 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,30 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#146820828 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,30 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-1839956566] 12:12:37,30 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#146820828] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,30 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-1839956566] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,30 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 12:12:37,32 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:37,32 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 12:12:37,32 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,32 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 12:12:37,32 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 12:12:37,32 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:12:37,32 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:37,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,32 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 12:12:37,32 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6d8b0d1d, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=null] 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testModuleShardLeaderMovement: 229 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 12:12:37,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 12:12:37,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,35 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:37,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:37,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:37,35 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testModuleShardLeaderMovement to state 12:12:37,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,36 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 12:12:37,36 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4750a499, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1444651685] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 12:12:37,37 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1444651685] 12:12:37,38 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 12:12:37,37 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 473.8 μs 12:12:37,38 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 12:12:37,38 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,38 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,38 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 90.51 μs 12:12:37,38 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,39 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 12:12:37,38 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1444651685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,39 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,39 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1d7345ff-dc8c-4fa5-b74f-b9f084b425b4}, logIndex: 0, isSendImmediate: true 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,40 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,41 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 12:12:37,41 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING]] 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 12:12:37,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,43 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 12:12:37,43 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 0 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:37,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 12:12:37,45 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 12:12:37,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1459788360] 12:12:37,45 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:37,45 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,46 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-908225933] 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 0 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:37,46 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 0 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 12:12:37,46 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 12:12:37,46 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,46 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:12:37,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 12:12:37,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 12:12:37,49 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,49 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020, primaryVersion=13] 12:12:37,51 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,51 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-2-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 12:12:37,51 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1474990232] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,52 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#2023034686 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,52 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1375245293] 12:12:37,52 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#2023034686] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,52 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 12:12:37,52 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1375245293] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 12:12:37,53 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:37,53 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 12:12:37,53 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,53 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:37,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,53 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 12:12:37,54 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:12:37,54 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 12:12:37,54 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 12:12:37,54 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:12:37,54 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2271f4e7, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=null] 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: 229 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 12:12:37,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,56 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,56 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement to state 12:12:37,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 12:12:37,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:37,58 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@355d3ed1, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:12:37,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:37,58 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 12:12:37,58 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#4458539] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 569.8 μs 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 12:12:37,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,62 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,62 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,62 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 12:12:37,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#4458539] 12:12:37,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 12:12:37,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 93.34 μs 12:12:37,63 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#4458539] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement] 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4bc63c04-6cac-45fa-b205-d95fe7434446}, logIndex: 0, isSendImmediate: true 12:12:37,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:37,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 12:12:37,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,70 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020 12:12:37,70 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING]] 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:37,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:37,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-908225933] 12:12:37,71 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:12:37,71 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:37,71 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 12:12:37,71 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1474990232] 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:37,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1474990232] 12:12:37,71 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 12:12:37,71 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 12:12:37,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 12:12:37,72 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:12:37,72 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 0 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:37,72 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,72 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:37,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 12:12:37,74 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 0 12:12:37,74 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:37,74 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 0 12:12:37,74 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 12:12:37,74 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702, primaryVersion=13] 12:12:37,74 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 12:12:37,75 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 12:12:37,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,77 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testModuleShardLeaderMovement, persistent : true 12:12:37,78 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 12:12:37,78 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-1661695928 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,78 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6fa51c77, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:37,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-1661695928] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testModuleShardLeaderMovement: 229 12:12:37,79 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 12:12:37,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:37,80 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 12:12:37,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 12:12:37,80 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:37,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:12:37,80 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 12:12:37,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 12:12:37,80 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 12:12:37,81 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 12:12:37,81 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-866724501] 12:12:37,81 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-866724501] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,81 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 12:12:37,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:37,81 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testModuleShardLeaderMovement to state 12:12:37,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,82 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@44f703ba, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:37,83 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 12:12:37,83 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-378737255] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 12:12:37,83 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,83 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 382.5 μs 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,83 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,84 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 12:12:37,84 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,84 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-378737255] 12:12:37,84 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 12:12:37,84 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-378737255] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,84 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 86.77 μs 12:12:37,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9beaba22-848e-4ef6-beb2-ec0f6cdb1065}, logIndex: 1, isSendImmediate: true 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 12:12:37,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 2 12:12:37,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:37,88 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702 12:12:37,88 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 12:12:37,87 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,88 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,88 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,88 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,88 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 1 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 12:12:37,88 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:37,88 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,88 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:37,88 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 12:12:37,88 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] 12:12:37,89 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,89 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:37,89 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,89 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,89 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 12:12:37,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:37,89 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 12:12:37,89 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,89 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,89 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,90 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,90 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 12:12:37,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 12:12:37,93 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,93 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1278494415] 12:12:37,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:12:37,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 12:12:37,94 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:12:37,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 12:12:37,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 12:12:37,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:12:37,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 12:12:37,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 12:12:37,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 12:12:37,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 12:12:37,95 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,96 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:12:37,96 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020, primaryVersion=13] 12:12:37,98 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,98 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 12:12:37,99 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 12:12:37,100 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-704938173] 12:12:37,100 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-151043940 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,100 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 12:12:37,100 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 12:12:37,100 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,100 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 12:12:37,100 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1471170849] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,100 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:12:37,100 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 12:12:37,101 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:37,101 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6a63f8dc, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 12:12:37,101 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,101 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: 229 12:12:37,101 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 12:12:37,101 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 12:12:37,101 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-3-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,101 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,102 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 12:12:37,102 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 12:12:37,102 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 12:12:37,102 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-151043940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,102 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-704938173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,102 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 12:12:37,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:12:37,103 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement to state 12:12:37,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,103 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 12:12:37,103 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ff32a10, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:12:37,104 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:12:37,104 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-229507562] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 12:12:37,105 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 358.0 μs 12:12:37,105 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 12:12:37,105 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,105 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-229507562] 12:12:37,105 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-229507562] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:37,106 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 12:12:37,106 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 12:12:37,106 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 12:12:37,106 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement] 12:12:37,106 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 101.4 μs 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=dbef05fc-e5f2-4c78-997a-effc9227c7a7}, logIndex: 1, isSendImmediate: true 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,106 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,107 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 12:12:37,107 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 2 12:12:37,108 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:37,109 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,109 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 1 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:37,109 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 12:12:37,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 12:12:37,111 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 12:12:37,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 12:12:37,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-999199020 12:12:37,111 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:12:37,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:12:37,111 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 12:12:37,111 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 12:12:37,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 12:12:37,111 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,112 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 12:12:37,112 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 12:12:37,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 12:12:37,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 12:12:37,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:37,112 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:37,112 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 12:12:37,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 12:12:37,112 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1278494415] 12:12:37,113 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:12:37,113 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,113 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 12:12:37,113 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 12:12:37,113 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 12:12:37,113 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:12:37,113 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1471170849] 12:12:37,113 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1471170849] 12:12:37,113 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,113 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2062831336] 12:12:37,113 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,113 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:37,113 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1459788360] 12:12:37,114 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:37,114 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] 12:12:37,114 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:37,114 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,115 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 12:12:37,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:12:37,116 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 12:12:37,116 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 12:12:37,116 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 12:12:37,116 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 12:12:37,116 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 12:12:37,116 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard cars, datastoreType Config 12:12:37,116 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2062831336] 12:12:37,116 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:37,116 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,117 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,117 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 12:12:37,118 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 12:12:37,119 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2062831336] 12:12:37,119 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:37,119 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,119 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,123 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1459788360] 12:12:37,123 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:37,123 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,123 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,123 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] 12:12:37,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:37,123 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,123 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,124 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard cars, datastoreType Config 12:12:37,124 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1459788360] 12:12:37,124 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:37,124 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,125 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,125 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 12:12:37,125 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,131 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-2-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/member-2-shard-cars-config_testModuleShardLeaderMovement$e]] 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e957ffd 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e957ffd 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,133 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,134 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,135 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,135 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,136 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,136 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,137 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 12:12:37,137 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 12:12:37,137 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,137 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,137 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 12:12:37,137 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 12:12:37,139 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:37,139 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,139 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,140 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 12:12:37,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,140 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:12:37,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,140 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:37,140 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,140 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 12:12:37,141 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,141 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 12:12:37,141 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 1 - switching to Follower 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,143 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 12:12:37,143 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,143 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:37,144 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:37,144 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:37,144 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,144 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 12:12:37,144 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,145 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,145 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,145 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,145 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,146 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,146 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,146 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 13.36 ms 12:12:37,146 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,146 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,146 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,146 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,146 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,146 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,147 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,147 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,148 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,151 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,151 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 12:12:37,151 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1459788360] 12:12:37,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:37,151 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,151 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#1579261810] 12:12:37,152 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2062831336] 12:12:37,152 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:37,152 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,159 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1672512702] 12:12:37,159 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] 12:12:37,159 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:37,159 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,159 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,160 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard cars, datastoreType Config 12:12:37,160 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] 12:12:37,160 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:37,160 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,160 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,160 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 12:12:37,160 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,161 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-3-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[akka://cluster-test@127.0.0.1:2557/temp/member-3-shard-cars-config_testModuleShardLeaderMovement$e]] 12:12:37,161 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 12:12:37,162 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e17edd 12:12:37,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e17edd 12:12:37,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 12:12:37,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 12:12:37,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,163 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,164 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,164 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 12:12:37,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 12:12:37,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 12:12:37,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 12:12:37,166 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:37,166 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,167 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,168 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:37,168 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 2 - updating term 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,168 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,168 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 12:12:37,168 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 2 - switching to Follower 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,169 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,169 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 12:12:37,170 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,170 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 12:12:37,170 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:12:37,170 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,171 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cannot append entries because sender's term 2 is less than 3 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,171 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 12:12:37,171 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,172 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-3-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-3-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,172 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,172 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,172 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 10.73 ms 12:12:37,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,172 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,172 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,174 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,174 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,174 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,174 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,174 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,174 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,175 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 12:12:37,175 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1606743638] 12:12:37,175 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:37,175 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,175 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#499662393] 12:12:37,175 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 12:12:37,176 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,176 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onShutDown 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 12:12:37,176 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@10eb66fc 12:12:37,176 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@10eb66fc 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 12:12:37,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 12:12:37,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 12:12:37,178 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,178 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,178 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:37,178 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,178 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,178 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,178 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,179 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,179 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 3 - switching to Follower 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 3 - updating term 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 12:12:37,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:12:37,180 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,181 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 12:12:37,181 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,181 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 12:12:37,181 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:12:37,181 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:12:37,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,182 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 12:12:37,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:37,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:37,182 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,182 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:37,182 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 12:12:37,182 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,183 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,183 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,183 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,183 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,183 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 7.280 ms 12:12:37,183 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer succeeded - sending PoisonPill 12:12:37,183 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,183 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,183 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,184 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,184 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,184 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,184 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:37,184 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:12:37,184 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:12:37,185 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 12:12:37,185 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,185 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,185 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,185 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 12:12:37,186 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 12:12:37,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,186 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,186 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:37,186 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 12:12:37,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:12:37,187 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:12:37,188 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:37,188 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:37,195 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:37,195 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:37,195 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:37,195 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:37,197 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:12:37,215 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 12:12:37,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,216 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 12:12:37,216 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onShutDown 12:12:37,216 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 12:12:37,216 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,217 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f26c1e0 12:12:37,217 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f26c1e0 12:12:37,217 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 12:12:37,217 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 12:12:37,217 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,217 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,220 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,220 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:37,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 12:12:37,222 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 12:12:37,222 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,222 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,222 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 12:12:37,222 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 12:12:37,223 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 12:12:37,223 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,223 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=5, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,223 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 5 12:12:37,223 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 12:12:37,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,224 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 5 12:12:37,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,224 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 12:12:37,224 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,224 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 4 - switching to Follower 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=5, votedFor=null 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 1 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=5, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 5 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 12:12:37,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 12:12:37,225 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 12:12:37,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 12:12:37,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 12:12:37,227 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 12:12:37,227 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,228 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,228 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 12:12:37,228 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:37,228 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 12:12:37,228 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 11.47 ms 12:12:37,228 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer succeeded - sending PoisonPill 12:12:37,228 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 12:12:37,229 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testModuleShardLeaderMovement 12:12:37,229 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:37,229 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:12:37,229 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:12:37,230 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 12:12:37,230 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 12:12:37,230 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:37,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,231 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 12:12:37,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 12:12:37,231 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testModuleShardLeaderMovement 12:12:37,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:37,232 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:12:37,232 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:12:37,233 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:37,233 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:37,733 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:12:38,202 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-7605167928901664707] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:12:38,249 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:38,249 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:38,252 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:12:38,252 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5724800490223156652] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [07/09/2025 12:12:38.255] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:38,268 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 12:12:38,268 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onShutDown 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74a17338 12:12:38,269 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74a17338 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:38,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:38,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onShutDown 12:12:38,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2fb48837 12:12:38,270 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2fb48837 12:12:38,270 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-config_testModuleShardLeaderMovement 12:12:38,270 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:42,833 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 12:12:42,834 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 12:12:42,834 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:12:42,835 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:42,835 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:42,836 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:42,836 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:45,61 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:45,550 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:12:47,461 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 12:12:47,462 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.193 s 12:12:47,462 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 12:12:47,462 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 12:12:47,462 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer expired 12:12:47,462 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 12:12:47,463 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 12:12:47,463 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:47,463 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:47,463 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 5 12:12:47,463 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:47,463 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 12:12:47,463 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 12:12:47,464 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testModuleShardLeaderMovement 12:12:47,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:47,464 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:12:47,464 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:12:47,465 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 12:12:47,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:47,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:47,466 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:12:47,466 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 12:12:47,466 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating leader transfer 12:12:47,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onShutDown 12:12:47,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4610354f 12:12:47,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4610354f 12:12:47,466 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 12:12:47,466 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7494760f 12:12:47,466 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7494760f 12:12:47,466 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 12:12:47,466 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 12:12:47,467 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:47,467 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testModuleShardLeaderMovement 12:12:47,467 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:47,467 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:51,132 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 12:12:51,132 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement] 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.667 s 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 12:12:51,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 12:12:51,134 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 12:12:51,134 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 12:12:51,134 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:12:51,135 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:12:51,136 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:51,136 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:51,141 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:12:51.144] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:51,162 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:51,176 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [109806894018026661] 12:12:51,176 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:12:51,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:12:51,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:12:51,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:51,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:51,193 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,194 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,195 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:12:51,196 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:12:51,196 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:12:51,201 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 12:12:51,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2043040144] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,215 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:12:51,228 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:12:51,228 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 12:12:51,229 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 12:12:51,229 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:51,229 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,229 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,235 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:51,236 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 12:12:51,237 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@750b43cf 12:12:51,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:51,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 12:12:51,241 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,245 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 12:12:51,245 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:51,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 12:12:51,246 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-492565167] 12:12:51,246 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardReplica, persistent : true 12:12:51,246 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,247 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#-277996946 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 12:12:51,247 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1046526191] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,248 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardReplica, persistent : true 12:12:51,248 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#1696541094 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 12:12:51,248 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#-1086457588] 12:12:51,249 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:12:51,249 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardReplica with journal batch size 1 12:12:51,250 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:12:51,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,250 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 12:12:51,250 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,251 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:51,251 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,251 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:12:51,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 12:12:51,252 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 12:12:51,253 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#-277996946] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,253 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-492565167] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,253 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#1696541094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,253 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#-1086457588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,254 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardReplica with journal batch size 1 12:12:51,254 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#161809970] 12:12:51,254 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#161809970] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,255 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,255 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 12:12:51,255 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 12:12:51,255 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,255 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,255 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:51,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:12:51,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 12:12:51,255 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 12:12:51,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@15cb808f 12:12:51,256 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 12:12:51,257 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:51,257 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,257 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardReplica, persistent : true 12:12:51,258 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardReplica with journal batch size 1 12:12:51,259 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-1071543227] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,259 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#450940961 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,259 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/shard-member-1:snapshot-read#-20090690] 12:12:51,259 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#450940961] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,259 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,259 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 12:12:51,259 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 12:12:51,259 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:51,260 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 12:12:51,260 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,260 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,260 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,260 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 12:12:51,260 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-1071543227] 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-1656171754] 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:12:51,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 12:12:51,262 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardReplica, persistent : true 12:12:51,262 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#2056013447 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 12:12:51,262 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/shard-member-1:snapshot-read#-573936847] 12:12:51,262 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardReplica with journal batch size 1 12:12:51,263 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:51,263 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 12:12:51,263 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,263 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,263 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:12:51,263 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 12:12:51,263 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,263 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 12:12:51,263 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 12:12:51,264 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:51,264 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 12:12:51,269 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 12:12:51,285 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5423643218864955693] 12:12:51,285 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:12:51,287 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:51,287 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:12:51,289 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:51,289 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:51,303 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,304 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,307 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 12:12:51,314 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-250582382] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,315 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:12:51,325 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:51,326 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:12:51,326 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 12:12:51,326 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 12:12:51,330 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2ef49c96 12:12:51,331 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 12:12:51,331 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:51,332 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 12:12:51,332 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,333 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,335 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 12:12:51,343 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 12:12:51,344 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 12:12:51,344 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1699739534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,344 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-712586000] 12:12:51,344 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-712586000] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,344 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 12:12:51,344 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1048991640] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,345 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 12:12:51,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-1500202607] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,345 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1048991640 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 12:12:51,345 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#1875835668] 12:12:51,345 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1756977022 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 12:12:51,355 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 12:12:51,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1756977022] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,355 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#1875835668] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,355 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-956707956] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,356 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:12:51,357 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,357 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,359 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:12:51,359 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 12:12:51,362 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:12:51,362 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,362 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,362 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 12:12:51,362 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:12:51,362 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 12:12:51,363 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,363 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 12:12:51,363 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 12:12:51,363 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:51,363 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 12:12:51,363 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 12:12:51,365 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 12:12:51,365 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,365 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 12:12:51,365 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 12:12:51,365 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:51,365 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 12:12:51,370 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 12:12:51,370 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52ab75e4 12:12:51,374 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:51,374 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 12:12:51,375 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#1565121466] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,375 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 12:12:51,375 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 12:12:51,376 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 12:12:51,376 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#788910106 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 12:12:51,377 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#788910106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,377 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#2138318973] 12:12:51,377 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,378 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,378 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,378 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 12:12:51,378 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 12:12:51,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:12:51,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 12:12:51,378 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:51,378 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 12:12:51,379 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 12:12:51,379 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-1810535933 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 12:12:51,379 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#12247052] 12:12:51,382 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 12:12:51,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,383 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:12:51,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 12:12:51,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 12:12:51,383 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 12:12:51,383 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:51,383 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 12:12:51,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:12:51,397 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:12:51,400 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:51,400 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,400 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:51,400 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,400 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,401 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,401 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:51,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1450014011360781294] 12:12:51,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:12:51,414 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:12:51,414 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:12:51,415 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:51,415 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:51,431 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,433 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,437 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 12:12:51,448 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:12:51,451 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1841709053] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,474 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:12:51,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 12:12:51,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 12:12:51,476 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:12:51,480 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:51,480 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,480 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,482 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:51,482 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:51,484 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@48bc9d78 12:12:51,484 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 12:12:51,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:51,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 12:12:51,486 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-3-shard-cars-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1546568369] 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:51,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-3-shard-people-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-858040065] 12:12:51,487 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:51,488 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardReplica, persistent : true 12:12:51,488 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1546568369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,488 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#909818436] 12:12:51,488 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#909818436] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,488 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardReplica with journal batch size 1 12:12:51,488 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-322205013] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,489 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-322205013 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 12:12:51,489 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardReplica, persistent : true 12:12:51,489 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-858040065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,489 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#-420546646] 12:12:51,489 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#-420546646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,489 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#-1198175848] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,489 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#-1198175848 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 12:12:51,490 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 12:12:51,492 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#928801741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,494 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardReplica with journal batch size 1 12:12:51,494 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,494 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:12:51,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 12:12:51,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardReplica is no longer the leader 12:12:51,495 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:51,495 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,495 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 12:12:51,496 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,496 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:51,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:12:51,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 12:12:51,496 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 12:12:51,496 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,496 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:12:51,496 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardReplica is no longer the leader 12:12:51,496 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 12:12:51,496 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:51,496 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:51,496 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 12:12:51,500 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:12:51,500 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 12:12:51,500 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 12:12:51,501 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:51,501 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,501 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,503 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 12:12:51,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3bd586be 12:12:51,506 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 12:12:51,507 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:51,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:51,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 12:12:51,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:51,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-3-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-2034919396] 12:12:51,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 12:12:51,508 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardReplica, persistent : true 12:12:51,509 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-2034919396] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,510 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/shard-member-3:snapshot-read#1528735763] 12:12:51,510 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardReplica with journal batch size 1 12:12:51,510 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,510 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:51,510 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-1008855787 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 12:12:51,510 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-1008855787] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:51,510 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-3-shard-people-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-1667212789] 12:12:51,511 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,511 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:12:51,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 12:12:51,511 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardReplica, persistent : true 12:12:51,512 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/shard-member-3:snapshot-read#-600321000] 12:12:51,512 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#788287871 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 12:12:51,513 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardReplica with journal batch size 1 12:12:51,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardReplica is no longer the leader 12:12:51,513 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 12:12:51,513 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:51,514 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:51,514 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:12:51,514 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,514 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 12:12:51,514 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardReplica is no longer the leader 12:12:51,514 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 12:12:51,514 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:51,514 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:51,514 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 12:12:51,514 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,515 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:51,515 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 12:12:51,579 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:51,579 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardReplica 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 12:12:51,580 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 12:12:51,580 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardReplica 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,619 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 12:12:51,619 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 12:12:51,620 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardReplica 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 12:12:51,629 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 12:12:51,629 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 12:12:51,659 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:12:51,659 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 12:12:51,659 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 12:12:51,659 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 12:12:51,660 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 12:12:51,660 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:12:51,660 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:51,660 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 12:12:51,660 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 12:12:51,660 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 12:12:51,959 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:51,959 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardReplica 12:12:51,960 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 12:12:51,960 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:51,960 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardReplica 12:12:51,960 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 12:12:52,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:52,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,39 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 12:12:52,39 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:52,39 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardReplica 12:12:52,39 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 12:12:52,190 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:12:52,190 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:12:52,191 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:52,192 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:52,192 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:52,193 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:52,193 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:52,201 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,201 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:52,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 12:12:52,319 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 12:12:52,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardReplica 12:12:52,320 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardReplica 12:12:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,323 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 12:12:52,325 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,325 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 12:12:52,326 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,328 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:52,328 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 12:12:52,328 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,328 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:52,328 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,328 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 12:12:52,329 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,329 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,329 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,329 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,379 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:52,379 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 12:12:52,379 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 12:12:52,379 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardReplica 12:12:52,379 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardReplica 12:12:52,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,382 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,383 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,383 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 12:12:52,383 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 12:12:52,383 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:52,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 12:12:52,385 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 12:12:52,386 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,386 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 12:12:52,386 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:52,386 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 12:12:52,386 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:52,388 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:52,388 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:52,388 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 12:12:52,388 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 12:12:52,388 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,388 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,388 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:52,388 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:52,388 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 12:12:52,388 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2043040144] 12:12:52,388 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:52,389 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:52,389 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:52,389 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,389 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-250582382] 12:12:52,389 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,389 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,389 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:52,389 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,389 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1699739534] 12:12:52,389 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1699739534] 12:12:52,390 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1841709053] 12:12:52,390 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:52,390 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1546568369] 12:12:52,390 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1546568369] 12:12:52,393 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-3, datastoreType Config 12:12:52,394 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 12:12:52,395 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:52,396 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 12:12:52,397 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 12:12:52,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:52,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,399 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 12:12:52,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardReplica 12:12:52,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'}, state: Idle 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,401 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 12:12:52,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:52,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:52,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=db18fbe8-f7e2-4457-a654-f937e2282cb2}, logIndex: 0, isSendImmediate: true 12:12:52,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:52,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardReplica 12:12:52,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 12:12:52,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'} 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:52,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 12:12:52,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Number of entries to be appended = 1 12:12:52,405 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:12:52,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 12:12:52,405 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 12:12:52,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:52,406 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, votingState=VOTING]] 12:12:52,406 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Log size is now 1 12:12:52,406 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:12:52,407 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:52,407 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,408 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,408 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 12:12:52,408 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,408 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,408 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:52,408 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 12:12:52,408 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 12:12:52,408 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: 0, nextIndex: 1 12:12:52,408 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:12:52,408 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Setting last applied to 0 12:12:52,408 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 12:12:52,408 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 12:12:52,408 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,408 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:52,409 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,409 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:52,409 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 12:12:52,409 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 12:12:52,409 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 12:12:52,409 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 12:12:52,409 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,409 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Commit index set to 0 12:12:52,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:12:52,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Setting last applied to 0 12:12:52,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 12:12:52,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 12:12:52,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 12:12:52,411 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2043040144] 12:12:52,412 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:52,412 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:52,412 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:52,412 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-250582382] 12:12:52,412 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:52,412 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1699739534] 12:12:52,412 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1699739534] 12:12:52,413 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1841709053] 12:12:52,413 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1546568369] with 27000 ms timeout 12:12:52,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onShutDown 12:12:52,413 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardReplica 12:12:52,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:52,415 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 12:12:52,415 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1546568369] 12:12:52,415 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 12:12:52,415 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:12:52,415 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 12:12:52,415 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:52,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node [akka://cluster-test@127.0.0.1:2559] as [Down] 12:12:52,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 12:12:52,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 12:12:52,416 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:12:52,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 12:12:52,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 12:12:52,416 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 12:12:52,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:52,416 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 12:12:52,417 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:52,417 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:52,417 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:12:52,417 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:12:52,417 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 12:12:52,418 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,418 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 12:12:52,418 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:12:52,418 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 12:12:52,418 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 12:12:52,418 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,418 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 12:12:52,418 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:52,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:52,418 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:12:52,418 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:12:52,419 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 12:12:52,420 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:52,420 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:52,421 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [akka://cluster-test@127.0.0.1:2558] 12:12:52,424 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 12:12:52,427 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Shutting down... 12:12:52,427 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Successfully shut down 12:12:52,431 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:52,431 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:52,432 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:52,434 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:52,436 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:12:52,440 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 12:12:52,440 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 12:12:52,440 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 12:12:52,440 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardReplica 12:12:52,441 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardReplica 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 12:12:52,442 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 12:12:52,444 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 12:12:52,444 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,445 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 12:12:52,445 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 12:12:52,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:52,447 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 12:12:52,448 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:52,448 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 12:12:52,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:52,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:52,463 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:12:52,478 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6455976795875629178] 12:12:52,479 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:12:52,483 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 12:12:52,483 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:12:52,485 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:12:52,485 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:12:52,501 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:52,504 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:52,506 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 12:12:52,514 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:12:52,516 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#572157725] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:12:52,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 12:12:52,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 12:12:52,532 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 12:12:52,533 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:52,535 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@769fa166 12:12:52,535 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:52,535 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 12:12:52,536 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:12:52,539 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 12:12:52,541 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 12:12:52,542 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 12:12:52,542 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 12:12:52,543 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 12:12:52,543 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-626898757] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,543 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#1211909004] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,543 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1891801850 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 12:12:52,543 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1891801850] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,543 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1728439970 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 12:12:52,543 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-239502408] 12:12:52,544 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 12:12:52,545 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:12:52,546 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:52,546 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,546 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:52,546 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 12:12:52,546 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:52,546 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:52,546 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:12:52,546 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received ReplicatedLogEntry for recovery: index: 0, size: 221 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=null, votingState=VOTING]] 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received apply journal entries for recovery, applying to state: 0 to 0 12:12:52,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 12:12:52,548 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:52,548 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in 283.4 μs - Switching actor to Follower - last log index = 0, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 1 12:12:52,548 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1383298410] 12:12:52,551 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1728439970] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,551 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-239502408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,551 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1773157651] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,551 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1383298410] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:52,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 12:12:52,552 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 12:12:52,552 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:52,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:12:52,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 12:12:52,552 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 12:12:52,554 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:12:52,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,555 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 12:12:52,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@552c1ee3 12:12:52,557 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 12:12:52,558 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 12:12:52,558 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:12:52,558 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,559 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 12:12:52,560 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 12:12:52,560 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-231938616] 12:12:52,560 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1673620158 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 12:12:52,560 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-478601208] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:52,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 12:12:52,561 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 12:12:52,561 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 12:12:52,561 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 12:12:52,561 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 12:12:52,561 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1855043253 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 12:12:52,562 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#290091382] 12:12:52,562 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 12:12:52,562 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 12:12:52,562 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1673620158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:12:52,563 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:52,563 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:52,563 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:12:52,563 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 12:12:52,563 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:52,564 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 12:12:52,564 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:12:52,564 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 12:12:52,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:12:52,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 12:12:52,564 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 12:12:52,564 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:52,564 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 12:12:52,564 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 12:12:52,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 12:12:52,643 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 12:12:52,644 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 12:12:52,721 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 12:12:52,722 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 12:12:52,761 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 12:12:52,762 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 12:12:52,963 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:53,42 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:53,83 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:53,210 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:12:53,282 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:53,362 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:53,402 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:53,435 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5423643218864955693] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:12:53,442 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5423643218864955693] but current is [6455976795875629178]. Reason: ActorSystem terminated 12:12:53,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:53,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 12:12:53,542 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = null 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 12:12:53,543 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#457215847]] intended for system uid [5423643218864955693], self uid is [6455976795875629178] 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = null 12:12:53,543 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:53,543 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 12:12:53,543 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:53,544 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-1953429140]] intended for system uid [5423643218864955693], self uid is [6455976795875629178] 12:12:53,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = null 12:12:53,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:53,544 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 12:12:53,544 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:53,544 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 12:12:53,544 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 12:12:53,544 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = null 12:12:53,544 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:53,545 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 12:12:53,545 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 12:12:53,545 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 12:12:53,546 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 12:12:53,546 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:53,547 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1137, lastApplied : -1, commitIndex : -1 12:12:53,547 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1218, lastApplied : -1, commitIndex : -1 12:12:53,547 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1135, lastApplied : 0, commitIndex : 0 12:12:53,549 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:53,549 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:53,603 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:53,682 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:53,723 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:53,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:53,921 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:53,923 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 375, lastApplied : -1, commitIndex : -1 12:12:53,923 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:53,981 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 432, lastApplied : 0, commitIndex : 0 12:12:54,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:54,0 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:54,1 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 454, lastApplied : -1, commitIndex : -1 12:12:54,2 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,41 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:54,41 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:54,42 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,42 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 495, lastApplied : -1, commitIndex : -1 12:12:54,242 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:54,242 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,302 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:54,321 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,322 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:54,362 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,362 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,520 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:54,562 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:54,562 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,622 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 12:12:54,642 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:54,642 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:54,681 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,682 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,882 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,884 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:54,941 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 12:12:54,962 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:54,963 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,1 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:55,2 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:55,202 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:55,203 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,231 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#457215847]] intended for system uid [5423643218864955693], self uid is [6455976795875629178] 12:12:55,262 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:55,281 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:55,281 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:55,321 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:55,323 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:55,522 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:55,522 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,541 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:55,582 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 12:12:55,601 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,601 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,643 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:55,643 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:55,843 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,844 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:55,903 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:55,921 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:55,922 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:55,962 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:55,962 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,162 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:56,162 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:56,223 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 12:12:56,241 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,242 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,282 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,282 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:56,483 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:56,483 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:56,541 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 12:12:56,561 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:56,561 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,562 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:56,602 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,602 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,802 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:56,802 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,861 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 12:12:56,881 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,881 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,921 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:56,922 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:57,122 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:57,123 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:57,181 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 12:12:57,201 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:57,202 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:57,242 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:57,242 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:57,250 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#457215847]] intended for system uid [5423643218864955693], self uid is [6455976795875629178] 12:12:57,439 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,439 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 12:12:57,439 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,439 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,439 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:12:57,440 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,440 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 12:12:57,440 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,442 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:57,442 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 12:12:57,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 12:12:57,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,443 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,443 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 12:12:57,443 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,443 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 12:12:57,443 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 12:12:57,443 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:57,445 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:57,499 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,500 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 12:12:57,500 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:57,500 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:12:57,500 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 12:12:57,502 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,502 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:57,503 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 12:12:57,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 12:12:57,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:12:57,503 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 12:12:57,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,504 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 12:12:57,504 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 12:12:57,504 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,520 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 12:12:57,520 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 12:12:57,522 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,522 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 12:12:57,523 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 12:12:57,523 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 12:12:57,523 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:57,559 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,560 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 12:12:57,560 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,560 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,560 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:12:57,560 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,560 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 12:12:57,560 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 12:12:57,562 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:57,562 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 12:12:57,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 12:12:57,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:12:57,563 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 12:12:57,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:12:57,563 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:57,563 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 12:12:57,564 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 12:12:57,564 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 12:12:57,580 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:57,762 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 12:12:57,764 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:57,821 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 12:12:57,841 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 12:12:57,841 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 12:12:57,882 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 12:12:57,882 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 12:12:58,81 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 12:12:58,82 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:58,141 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:58,161 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:58,161 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:58,203 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,203 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,310 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Down)]. 12:12:58,311 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,312 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,328 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,328 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,402 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,402 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,462 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:58,482 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,482 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,521 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:58,521 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:58,557 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2559] 12:12:58,558 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,558 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,559 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5423643218864955693] but current is [6455976795875629178]. Reason: Cluster member removed, previous status [Down] 12:12:58,600 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:12:58,679 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,679 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:12:58,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5423643218864955693] but current is [6455976795875629178]. Reason: Cluster member removed, previous status [Down] 12:12:58,721 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:58,721 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:58,781 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 12:12:58,801 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:58,801 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:58,842 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:58,842 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:59,43 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:59,43 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:12:59,101 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 12:12:59,121 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:59,121 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:59,161 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:59,161 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:59,362 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:12:59,362 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:59,421 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 12:12:59,441 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:59,441 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:59,482 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:59,482 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:12:59,620 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:12:59,625 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:59,626 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:12:59,627 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:59,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#572157725] 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 12:12:59,627 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:12:59,627 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 12:12:59,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 12:12:59,628 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-626898757] 12:12:59,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-626898757] 12:12:59,629 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-2, datastoreType Config 12:12:59,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 12:12:59,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:12:59,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5ade466a] 12:12:59,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060 for shard member-2-shard-cars-config_testRemoveShardReplica 12:12:59,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'}, state: Idle 12:12:59,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 12:12:59,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:59,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:59,631 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'} 12:12:59,631 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 12:12:59,631 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 12:12:59,631 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 12:12:59,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2043040144] 12:12:59,631 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:12:59,631 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:59,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1752965060] 12:12:59,632 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#572157725] 12:12:59,632 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:59,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 12:12:59,632 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-626898757] 12:12:59,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from 0 to 1 12:12:59,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 12:12:59,632 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:12:59,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=1 12:12:59,633 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-626898757] with 27000 ms timeout 12:12:59,633 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 12:12:59,633 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 12:12:59,633 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 12:12:59,634 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:12:59,634 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 12:12:59,634 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,634 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 12:12:59,634 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-626898757] 12:12:59,681 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:59,682 PM [cluster-test-shard-dispatcher-24] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:12:59,682 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#572157725] 12:12:59,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:12:59,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 12:12:59,682 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 12:12:59,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 12:12:59,683 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 12:12:59,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 12:12:59,683 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 12:12:59,684 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,684 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:12:59,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:12:59,685 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 12:12:59,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 12:12:59,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 12:12:59,685 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:12:59,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 12:12:59,685 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 12:12:59,686 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 12:12:59,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,686 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 12:12:59,686 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,686 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:12:59,686 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:12:59,688 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:59,688 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:12:59,695 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:59,695 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:59,695 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:59,695 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:12:59,698 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:12:59,710 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:12:59,710 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 12:12:59,710 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 12:12:59,711 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onShutDown 12:12:59,712 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardReplica 12:12:59,712 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,712 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:12:59,712 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:12:59,713 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 12:12:59,714 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 12:12:59,714 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 12:12:59,714 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onShutDown 12:12:59,714 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:12:59,714 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onShutDown 12:12:59,715 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardReplica 12:12:59,715 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,715 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardReplica 12:12:59,715 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:12:59,715 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:12:59,716 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:12:59,717 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:12:59,717 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:00,596 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:00,709 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6455976795875629178] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:00,736 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:00,736 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:00,738 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:00.740] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:00,741 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [1450014011360781294] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:00,745 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 12:13:00,746 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 12:13:00,746 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 12:13:00,746 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:13:00,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onShutDown 12:13:00,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@14f59f8b 12:13:00,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@14f59f8b 12:13:00,746 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onShutDown 12:13:00,746 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Initiating leader transfer 12:13:00,746 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 12:13:00,746 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69ab926b 12:13:00,747 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardReplica 12:13:00,747 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69ab926b 12:13:00,747 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 12:13:00,747 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Attempting to transfer leadership 12:13:00,747 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:00,747 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:13:00,747 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:01,279 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 12:13:01,279 PM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 533.1 ms 12:13:01,280 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 12:13:01,280 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: leader transfer failed - sending PoisonPill 12:13:01,280 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Leadership transfer expired 12:13:01,280 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardReplica 12:13:01,280 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 12:13:01,281 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:01,281 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:01,282 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 12:13:01,282 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 12:13:01,282 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 12:13:01,282 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:01,282 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onShutDown 12:13:01,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onShutDown 12:13:01,282 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Initiating leader transfer 12:13:01,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Initiating leader transfer 12:13:01,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@241191dc 12:13:01,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@241191dc 12:13:01,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7204dc70 12:13:01,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7204dc70 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 12:13:01,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:01,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 12:13:01,283 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 12:13:01,679 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 12:13:01,679 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 396.5 ms 12:13:01,679 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 12:13:01,679 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 12:13:01,679 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Leadership transfer expired 12:13:01,680 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardReplica 12:13:01,680 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:13:01,719 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 12:13:01,719 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 436.8 ms 12:13:01,719 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 12:13:01,719 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 12:13:01,719 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Leadership transfer expired 12:13:01,720 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardReplica 12:13:01,720 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 12:13:01,720 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:01,721 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:01,722 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:01,722 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:01,726 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:01.729] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:01,755 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:13:01,785 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4450700533932068888] 12:13:01,790 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:01,794 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:01,794 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:01,794 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:01,794 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:01,808 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:01,808 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:01,808 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:01,809 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,810 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,813 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 12:13:01,819 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1842010176] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,820 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:01,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:01,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:01,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:01,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3d4787f5 12:13:01,825 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 12:13:01,826 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,827 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,829 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 12:13:01,829 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:01,829 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,830 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1472000835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,830 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:01,830 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,830 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,830 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1108745927] 12:13:01,830 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,831 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveAllShardReplicas, persistent : true 12:13:01,831 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1108745927] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,831 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1744251651] 12:13:01,831 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,831 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,831 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1792758820] 12:13:01,831 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveAllShardReplicas, persistent : true 12:13:01,832 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-796662093 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,832 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 12:13:01,832 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:01,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,833 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-796662093] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,833 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,833 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#2077437251 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,833 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:01,833 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,833 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,834 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1744251651] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,834 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1792758820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,834 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#2077437251] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,835 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1824274026] 12:13:01,835 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1824274026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,835 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 12:13:01,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,836 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:01,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 12:13:01,836 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,836 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 12:13:01,836 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,836 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:01,836 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 12:13:01,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,838 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,838 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:01,838 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 12:13:01,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 12:13:01,838 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,838 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:01,839 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 12:13:01,840 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 12:13:01,840 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7d71e8bf 12:13:01,841 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 12:13:01,841 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:01,841 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-182984558] 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,842 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#1790887592] 12:13:01,842 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveAllShardReplicas, persistent : true 12:13:01,843 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 12:13:01,843 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 12:13:01,843 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#974228889 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,843 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#2001214561] 12:13:01,843 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#537368534 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#402482218] 12:13:01,843 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-182984558] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,843 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#1790887592] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 12:13:01,845 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:01,845 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:01,845 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 12:13:01,845 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 12:13:01,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,845 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 12:13:01,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 12:13:01,845 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,845 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 12:13:01,845 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 12:13:01,848 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:01,857 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8550129108600293928] 12:13:01,857 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:01,865 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:01,865 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:01,866 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:01,866 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:01,882 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,885 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,887 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 12:13:01,900 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1986263986] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,900 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:01,904 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:01,904 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:01,904 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 12:13:01,906 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:01,911 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 12:13:01,912 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,913 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@49452a9c 12:13:01,913 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:01,913 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:01,914 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 12:13:01,914 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:01,914 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveAllShardReplicas, persistent : true 12:13:01,915 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 12:13:01,915 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveAllShardReplicas, persistent : true 12:13:01,916 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 12:13:01,916 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1975312597 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:01,916 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#1009757061 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 12:13:01,916 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1404284565] 12:13:01,917 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 12:13:01,917 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#934892938] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,917 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1963309160] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,917 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1975312597] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,917 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#1009757061] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,917 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1404284565] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,920 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:01,923 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1079169074] 12:13:01,924 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1947689377] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,924 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1079169074] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,926 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,926 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,926 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,926 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 12:13:01,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:01,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 12:13:01,927 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,927 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:01,927 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 12:13:01,927 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:01,927 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,928 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:01,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,928 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,928 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:13:01,928 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 12:13:01,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveAllShardReplicas is no longer the leader 12:13:01,928 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,928 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:01,928 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 12:13:01,929 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 12:13:01,929 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1869f653 12:13:01,929 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 12:13:01,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:01,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,932 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 12:13:01,932 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#-472151946] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,932 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#1983574929 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,932 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#1983574929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:01,932 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 12:13:01,932 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-713919221] 12:13:01,933 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:01,933 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,933 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 12:13:01,934 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveAllShardReplicas, persistent : true 12:13:01,934 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:01,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 12:13:01,934 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 12:13:01,934 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 12:13:01,934 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-140781179 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:01,935 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-850220238] 12:13:01,935 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 12:13:01,936 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:01,936 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:01,936 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:13:01,936 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 12:13:01,936 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:01,936 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 12:13:01,936 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 12:13:01,936 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 12:13:01,936 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 12:13:01,950 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:13:01,952 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:01,956 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:01,956 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:01,957 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:01,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4618701969827466002] 12:13:01,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:01,978 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:01,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:01,979 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:01,979 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:02,11 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:02,14 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:02,16 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 12:13:02,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#506322045] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,24 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:02,35 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:02,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,37 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:02,39 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1fec1a0 12:13:02,40 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 12:13:02,41 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:02,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:02,44 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,45 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,46 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveAllShardReplicas, persistent : true 12:13:02,46 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1151631891] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,47 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveAllShardReplicas, persistent : true 12:13:02,47 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-153853424] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,47 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#-1910373973 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,47 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1761508758] 12:13:02,47 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#-1910373973] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,47 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1761508758] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,49 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 12:13:02,52 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-850808417] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,52 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 12:13:02,52 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-850808417 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,52 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:02,54 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:02,55 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:02,56 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:02,56 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,56 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,57 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:02,57 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:02,57 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,57 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 12:13:02,57 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:02,57 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,58 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:02,58 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:02,58 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,58 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveAllShardReplicas is no longer the leader 12:13:02,58 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:02,61 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1734517227] 12:13:02,61 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1734517227] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,61 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 12:13:02,62 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1268413873] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,63 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:02,64 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:02,64 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,67 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 12:13:02,68 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@75686869 12:13:02,68 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 12:13:02,69 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:02,69 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:02,69 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,70 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 12:13:02,70 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2110702630] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,70 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1386710090] 12:13:02,70 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-551445132 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,70 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 12:13:02,70 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-551445132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:02,70 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,71 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2110702630] 12:13:02,71 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:02,71 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:02,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 12:13:02,71 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,71 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 12:13:02,72 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveAllShardReplicas, persistent : true 12:13:02,72 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#1107438405 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,72 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-2041440333] 12:13:02,72 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 12:13:02,72 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,72 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,72 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-382592989] 12:13:02,73 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:02,73 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 12:13:02,73 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,73 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 12:13:02,73 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:02,73 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 12:13:02,73 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:13:02,73 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 12:13:02,73 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:02,73 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:02,73 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 12:13:02,73 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 12:13:02,73 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 12:13:02,73 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 12:13:02,73 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,168 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 12:13:02,169 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 12:13:02,169 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,198 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 12:13:02,199 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 12:13:02,199 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 12:13:02,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:02,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 12:13:02,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 12:13:02,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 12:13:02,237 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 12:13:02,237 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:02,237 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 12:13:02,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:02,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,238 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 12:13:02,238 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 12:13:02,238 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 12:13:02,238 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 12:13:02,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:02,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,507 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 12:13:02,608 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:02,608 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,608 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 12:13:02,618 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:02,618 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,618 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 12:13:02,647 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:02,647 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 12:13:02,647 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 12:13:02,808 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:02,808 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:02,809 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:02,810 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,810 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,817 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,818 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:02,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,868 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 12:13:02,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:02,870 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:02,872 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:02,872 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:02,872 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 12:13:02,872 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,872 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,873 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:02,873 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 12:13:02,873 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 12:13:02,873 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 12:13:02,873 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:02,873 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 12:13:02,874 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,874 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:02,874 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 12:13:02,874 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 12:13:02,874 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:02,874 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:02,875 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:02,875 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,875 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:02,875 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:02,902 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 12:13:02,987 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:02,988 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,988 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 12:13:02,988 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:02,988 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:02,989 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:02,989 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:02,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:02,990 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:02,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:02,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:02,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:02,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,991 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:02,991 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:02,991 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:02,991 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 12:13:02,992 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:02,992 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 12:13:02,993 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:02,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 12:13:02,993 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:02,993 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:02,994 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 12:13:02,995 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:13:02,995 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:02,995 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:02,995 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:02,995 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:03,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,18 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 12:13:03,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,19 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:03,19 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:03,19 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:03,19 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:03,19 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:03,19 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:03,19 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:03,19 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:03,19 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:03,19 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:03,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:03,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,20 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:03,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:03,20 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,20 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 12:13:03,21 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 12:13:03,21 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,22 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 12:13:03,22 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 12:13:03,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,22 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,23 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 12:13:03,23 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 12:13:03,23 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,23 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 12:13:03,23 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,23 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 12:13:03,23 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:03,23 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,23 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:03,23 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,57 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 12:13:03,57 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 12:13:03,57 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 12:13:03,57 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveAllShardReplicas 12:13:03,58 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveAllShardReplicas 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 12:13:03,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:03,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:03,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:03,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:03,59 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:03,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 12:13:03,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:03,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 12:13:03,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:03,61 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:03,61 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:03,61 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 12:13:03,61 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,61 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,61 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:03,62 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 12:13:03,62 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 12:13:03,62 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 12:13:03,62 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,62 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 12:13:03,62 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 12:13:03,63 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 12:13:03,63 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1842010176] 12:13:03,63 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:03,63 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 12:13:03,63 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 12:13:03,63 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 12:13:03,63 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,63 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:03,63 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 12:13:03,63 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1108745927] 12:13:03,64 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1108745927] 12:13:03,64 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,64 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 12:13:03,64 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1986263986] 12:13:03,64 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:03,64 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:03,64 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#934892938] 12:13:03,64 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,64 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#934892938] 12:13:03,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#506322045] 12:13:03,65 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:03,65 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1151631891] 12:13:03,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1151631891] 12:13:03,65 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:03,65 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,76 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@660160a2] 12:13:03,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 12:13:03,78 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-pets-config_testRemoveAllShardReplicas, persistent : true 12:13:03,79 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#209726499 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:03,79 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#1604531025] 12:13:03,79 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 12:13:03,80 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:03,80 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:03,80 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 12:13:03,80 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 12:13:03,80 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:03,80 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 12:13:03,80 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:03,80 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:03,80 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 12:13:03,88 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@1ee804bd] 12:13:03,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 12:13:03,90 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-pets-config_testRemoveAllShardReplicas, persistent : true 12:13:03,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@74782d2a] 12:13:03,92 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 12:13:03,92 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1096431979] 12:13:03,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 12:13:03,92 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#-734211548 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 12:13:03,93 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1842010176] 12:13:03,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:03,93 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 12:13:03,93 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:03,93 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 12:13:03,93 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 12:13:03,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:03,93 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#108986902] 12:13:03,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 12:13:03,94 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#108986902] 12:13:03,94 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-pets-config_testRemoveAllShardReplicas, persistent : true 12:13:03,94 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:03,94 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:03,94 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 12:13:03,94 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1986263986] 12:13:03,94 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 12:13:03,94 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#1919942395] 12:13:03,94 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#1919942395] 12:13:03,94 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#506322045] 12:13:03,94 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 12:13:03,95 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 12:13:03,95 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1693462313] 12:13:03,95 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-1212358739 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 12:13:03,96 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:03,96 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:03,96 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:03,96 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 12:13:03,96 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 12:13:03,95 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 12:13:03,100 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 12:13:03,100 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 12:13:03,100 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 12:13:03,100 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#847475775] 12:13:03,101 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 12:13:03,101 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#847475775] 12:13:03,101 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 12:13:03,102 PM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 12:13:03,102 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 12:13:03,102 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 12:13:03,103 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 12:13:03,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 12:13:03,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 12:13:03,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 12:13:03,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 12:13:03,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 12:13:03,103 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 12:13:03,104 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 12:13:03,104 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,104 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 12:13:03,104 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:03,104 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:03,104 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 12:13:03,105 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'}, state: Idle 12:13:03,105 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'}, state: Idle 12:13:03,105 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]] 12:13:03,105 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'}, state: Idle 12:13:03,105 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,105 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]] 12:13:03,105 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'}, state: Idle 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]] 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=e6be4b3e-bb96-4e28-a3ab-e6f35359d211}, logIndex: 0, isSendImmediate: true 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9d5e4e15-ce86-4e96-b5b2-69c0499cf733}, logIndex: 0, isSendImmediate: true 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'} 12:13:03,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'} 12:13:03,106 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f0bdc9a6-4c05-4c5c-ba68-3a5a42753aac}, logIndex: 0, isSendImmediate: true 12:13:03,107 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,108 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,106 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]] 12:13:03,108 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,108 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,108 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,108 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'} 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=2ed0aef9-5488-4c3c-ac2d-2580575a2867}, logIndex: 0, isSendImmediate: true 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testRemoveAllShardReplicas 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,108 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'} 12:13:03,109 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,109 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, votingState=VOTING]] 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Log size is now 1 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, votingState=VOTING]] 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 12:13:03,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:03,110 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, votingState=VOTING]] 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 12:13:03,111 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, votingState=VOTING]] 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Log size is now 1 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 12:13:03,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:13:03,111 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 12:13:03,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,111 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 12:13:03,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 12:13:03,112 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 12:13:03,112 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:03,112 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,112 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,112 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:03,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:03,114 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,114 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,114 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-153853424] with 27000 ms timeout 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 12:13:03,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, cars] 12:13:03,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,115 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onShutDown 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-382592989] with 27000 ms timeout 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1151631891] with 27000 ms timeout 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 12:13:03,115 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onShutDown 12:13:03,115 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onShutDown 12:13:03,116 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveAllShardReplicas 12:13:03,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,116 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveAllShardReplicas 12:13:03,116 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveAllShardReplicas 12:13:03,116 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,116 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,116 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-153853424] 12:13:03,116 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1151631891] 12:13:03,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,116 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,117 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-382592989] 12:13:03,119 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2110702630] with 27000 ms timeout 12:13:03,119 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 12:13:03,119 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 12:13:03,119 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 12:13:03,119 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:03,120 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 12:13:03,120 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:03,120 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:03,120 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 12:13:03,120 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 12:13:03,120 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,120 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 12:13:03,120 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,120 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2110702630] 12:13:03,447 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-pets-config_testRemoveAllShardReplicas 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-pets-config_testRemoveAllShardReplicas 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 12:13:03,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 12:13:03,449 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 12:13:03,450 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 12:13:03,450 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:03,450 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:03,450 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:03,451 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:03,451 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 12:13:03,451 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:03,451 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 12:13:03,451 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:03,451 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:03,451 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=1, voteGranted=true], my term is 1 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=1, voteGranted=true], current voteCount: 1 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 12:13:03,453 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 12:13:03,454 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 12:13:03,454 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,454 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 12:13:03,454 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 12:13:03,455 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-pets-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:03,455 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,455 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 12:13:03,455 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 12:13:03,455 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 12:13:03,455 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 12:13:03,455 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:03,455 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,455 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-pets-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:03,455 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 12:13:03,456 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 12:13:03,456 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 12:13:03,456 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 12:13:03,456 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:03,456 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 12:13:03,457 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'}, state: Idle 12:13:03,457 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]] 12:13:03,457 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,457 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,457 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=797656ef-a24c-4557-b379-d92c744555a8}, logIndex: 0, isSendImmediate: true 12:13:03,458 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,458 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testRemoveAllShardReplicas 12:13:03,458 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,458 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'} 12:13:03,458 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 12:13:03,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 12:13:03,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:03,459 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 12:13:03,459 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 12:13:03,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,459 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 12:13:03,459 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 12:13:03,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, votingState=VOTING]] 12:13:03,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Log size is now 1 12:13:03,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1842010176] 12:13:03,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:03,461 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:03,461 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1108745927] 12:13:03,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1108745927] 12:13:03,461 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 12:13:03,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1842010176] 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:13:03,461 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 12:13:03,461 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1792758820] 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,461 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:03,462 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:03,462 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1792758820] 12:13:03,462 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 12:13:03,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:03,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 12:13:03,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 12:13:03,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 12:13:03,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 12:13:03,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1842010176] 12:13:03,463 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 12:13:03,463 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#847475775] with 27000 ms timeout 12:13:03,463 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#108986902] 12:13:03,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#108986902] 12:13:03,463 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 12:13:03,463 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 12:13:03,463 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onShutDown 12:13:03,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1986263986] 12:13:03,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:03,464 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#934892938] 12:13:03,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#934892938] 12:13:03,464 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-pets-config_testRemoveAllShardReplicas 12:13:03,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1986263986] 12:13:03,464 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:13:03,464 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1963309160] 12:13:03,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1963309160] 12:13:03,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1986263986] 12:13:03,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 12:13:03,464 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:03,464 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#1919942395] 12:13:03,464 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#847475775] 12:13:03,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#1919942395] 12:13:03,464 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 12:13:03,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#506322045] 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 12:13:03,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#506322045] 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 12:13:03,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#506322045] 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 12:13:03,465 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 12:13:03,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:03,465 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:03,466 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 12:13:03,466 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 12:13:03,466 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:03,466 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:03,467 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:03,467 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:03,472 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:03,472 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:03,472 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:03,472 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:03,475 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:03,485 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 12:13:03,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onShutDown 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onShutDown 12:13:03,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onShutDown 12:13:03,486 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testRemoveAllShardReplicas 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveAllShardReplicas 12:13:03,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,486 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveAllShardReplicas 12:13:03,486 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,486 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,487 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:03,487 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:03,487 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 12:13:03,487 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,487 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:03,487 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:13:03,488 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 12:13:03,488 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onShutDown 12:13:03,488 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 12:13:03,488 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,488 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveAllShardReplicas 12:13:03,488 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:03,488 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:03,488 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:03,489 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:03,489 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:03,828 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:04,488 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [4618701969827466002] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:04,507 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8550129108600293928] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:04,509 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:04,511 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:04,514 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:04.516] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:04,525 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 12:13:04,525 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onShutDown 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Initiating leader transfer 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onShutDown 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Initiating leader transfer 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onShutDown 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Initiating leader transfer 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b9653f9 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60c38db3 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b9653f9 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60c38db3 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c9b8a0c 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c9b8a0c 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 12:13:04,526 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:04,526 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 12:13:04,526 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 12:13:04,908 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 12:13:04,908 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 382.0 ms 12:13:04,908 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 12:13:04,908 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 12:13:04,908 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 12:13:04,909 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveAllShardReplicas 12:13:04,909 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 451.8 ms 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveAllShardReplicas 12:13:04,978 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:05,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 12:13:05,48 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 522.1 ms 12:13:05,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 12:13:05,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 12:13:05,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 12:13:05,49 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testRemoveAllShardReplicas 12:13:05,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:05,49 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:05,50 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:05,50 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 12:13:05,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:05,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:05,55 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:05,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 12:13:05,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Initiating leader transfer 12:13:05,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onShutDown 12:13:05,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Initiating leader transfer 12:13:05,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@49d46766 12:13:05,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@49d46766 12:13:05,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 12:13:05,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12928b59 12:13:05,56 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 12:13:05,56 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12928b59 12:13:05,56 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 12:13:05,56 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 12:13:05,56 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:05,56 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:05,56 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 12:13:05,56 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 12:13:05,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 12:13:05,427 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 371.7 ms 12:13:05,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 12:13:05,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 12:13:05,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 12:13:05,428 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveAllShardReplicas 12:13:05,428 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:05,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 12:13:05,578 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 522.3 ms 12:13:05,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 12:13:05,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 12:13:05,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 12:13:05,579 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 12:13:05,579 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 12:13:05,579 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:05,580 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:05,581 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:05,581 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:05,585 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:05.587] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:05,611 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:05,622 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7374725983756734643] 12:13:05,622 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:05,626 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:05,626 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:05,627 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:05,627 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:05,649 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:05,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:05,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:05,652 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,656 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 12:13:05,660 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1261350602] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,662 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:05,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:05,663 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:05,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,668 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 12:13:05,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@74b2a6b3 12:13:05,668 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 12:13:05,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:05,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,669 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,669 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#286879409] 12:13:05,670 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplicaFailures, persistent : true 12:13:05,670 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,671 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplicaFailures with journal batch size 1 12:13:05,671 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#286879409] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,672 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#1980068337] 12:13:05,672 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#390604406] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,673 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:05,673 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:05,673 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:05,673 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 12:13:05,673 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,673 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 12:13:05,673 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 12:13:05,675 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#1980068337] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,675 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#1957042794] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,675 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:05,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 12:13:05,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 12:13:05,676 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:05,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 12:13:05,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 12:13:05,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:05,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:05,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#390604406 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 12:13:05,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 12:13:05,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 12:13:05,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 12:13:05,679 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderPayloadVersion=13] 12:13:05,679 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 12:13:05,679 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 12:13:05,681 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 12:13:05,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5aaef28a 12:13:05,682 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 12:13:05,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:05,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,683 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-278518465] 12:13:05,683 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplicaFailures, persistent : true 12:13:05,684 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-278518465] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,684 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#1100565265 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,684 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#1100565265] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,684 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#-1695434617] 12:13:05,684 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplicaFailures with journal batch size 1 12:13:05,684 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#-1695434617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,685 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard people 12:13:05,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 12:13:05,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 12:13:05,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:05,685 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:05,685 PM [cluster-test-akka.actor.default-dispatcher-18] [ERROR] ClusterAdminRpcService - Failed to add replica for shard people java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1188) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:241) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 12:13:05,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 12:13:05,686 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 12:13:05,686 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 12:13:05,686 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,686 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 12:13:05,686 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 12:13:05,687 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 12:13:05,687 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onShutDown 12:13:05,687 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2235b035 12:13:05,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 12:13:05,687 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2235b035 12:13:05,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 12:13:05,687 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:05,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 12:13:05,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 12:13:05,687 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 12:13:05,687 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplicaFailures 12:13:05,687 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 12:13:05,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Aborting 0 pending queued transactions 12:13:05,687 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderPayloadVersion=13] 12:13:05,688 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 12:13:05,688 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 12:13:05,688 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:05,688 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:05,688 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 12:13:05,688 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1261350602] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1261350602] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,688 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,688 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1261350602] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1261350602] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onShutDown 12:13:05,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5b67b321 12:13:05,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5b67b321 12:13:05,689 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplicaFailures 12:13:05,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Aborting 0 pending queued transactions 12:13:05,689 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 12:13:05,689 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:05,689 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:05,690 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:05,690 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:05,691 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:05,704 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:05,717 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3181504509661409951] 12:13:05,717 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:05,719 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:05,719 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:05,723 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:05,723 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:05,734 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:05,734 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:05,734 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:05,735 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,736 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,738 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 12:13:05,743 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1553541589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,744 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:05,749 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:05,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 12:13:05,749 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:05,749 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:05,751 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:05,752 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 12:13:05,752 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2caa75d0 12:13:05,752 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:05,752 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 12:13:05,753 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplica, persistent : true 12:13:05,754 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplica with journal batch size 1 12:13:05,754 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:05,754 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:05,754 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074] 12:13:05,754 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-818118927 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 12:13:05,754 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#-88470943] 12:13:05,754 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,754 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-818118927] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,755 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#-88470943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:05,756 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,756 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 12:13:05,756 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplica 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,756 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 12:13:05,756 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Received ElectionTimeout 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Election: Leader has following peers: [] 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 12:13:05,757 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 12:13:05,757 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 12:13:05,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 12:13:05,802 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1553541589] 12:13:05,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:05,803 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074] 12:13:05,803 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,806 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,809 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 12:13:05,810 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1106890079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,810 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:05,813 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:05,813 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 12:13:05,814 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:05,814 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:05,818 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:05,819 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@26521ea0 12:13:05,819 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 12:13:05,820 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:05,820 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 12:13:05,820 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:05,820 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:05,820 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:05,821 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplica, persistent : true 12:13:05,821 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,821 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1449201871 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 12:13:05,821 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1449201871] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,821 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-172072835] 12:13:05,821 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplica with journal batch size 1 12:13:05,822 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-172072835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:05,822 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:05,822 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:05,822 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 12:13:05,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 12:13:05,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 12:13:05,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplica 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 12:13:05,823 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Received ElectionTimeout 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Election: Leader has following peers: [] 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:05,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 12:13:05,823 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 12:13:05,823 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 12:13:05,824 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 12:13:05,824 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 12:13:05,824 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:05,870 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1106890079] 12:13:05,870 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:05,870 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:05,887 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:13:05,952 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-992355537910435549] 12:13:05,955 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:05,961 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:05,961 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:05,964 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:05,964 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:05,977 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,979 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:05,985 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 12:13:05,991 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-835465265] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:05,992 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:06,3 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:06,7 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:06,8 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 12:13:06,9 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:06,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3a7a1654 12:13:06,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:06,12 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:06,15 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 12:13:06,18 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:06,18 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#640563460] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,20 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:06,21 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:06,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7b471c6f 12:13:06,22 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 12:13:06,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:06,34 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:06,37 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:06,41 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:06,41 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:06,41 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 12:13:06,41 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,732 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:06,733 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:06,733 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 12:13:06,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:06,734 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,734 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:06,734 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 12:13:06,734 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:06,734 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 12:13:06,825 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:13:06,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 12:13:06,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:06,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 12:13:06,828 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:13:06,828 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074, primaryVersion=13] 12:13:06,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074 for shard member-2-shard-cars-config_testAddShardReplica 12:13:06,832 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddShardReplica, persistent : true 12:13:06,832 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,832 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-91729521 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 12:13:06,832 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#-1825100629] 12:13:06,833 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-91729521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,833 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#-1825100629] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,833 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddShardReplica with journal batch size 1 12:13:06,833 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 12:13:06,833 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:06,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:13:06,834 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 12:13:06,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 1 12:13:06,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:13:06,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-config_testAddShardReplica 12:13:06,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:06,834 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:06,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:06,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:06,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 12:13:06,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 12:13:06,835 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 12:13:06,835 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 12:13:06,835 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7711c5ef, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=null] 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddShardReplica: 229 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-2-shard-cars-config_testAddShardReplica), Chunk: 1/1 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 12:13:06,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 12:13:06,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 12:13:06,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 12:13:06,836 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:13:06,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:06,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:13:06,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:13:06,836 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 12:13:06,837 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddShardReplica to state 12:13:06,837 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5065ab42, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to -1 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:06,837 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:13:06,838 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 12:13:06,838 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-1920910159] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 12:13:06,838 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-1920910159] 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 273.6 μs 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 12:13:06,838 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 12:13:06,838 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 12:13:06,839 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: SnapshotComplete received 12:13:06,839 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-1920910159] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,839 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 119.1 μs 12:13:06,839 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a journal response 12:13:06,839 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a snapshot response 12:13:06,839 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 12:13:06,839 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:13:06,839 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddShardReplica] 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8d74b528-dfd0-457e-bdcc-002875633ced}, logIndex: 0, isSendImmediate: true 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddShardReplica 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 12:13:06,840 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:06,842 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074 12:13:06,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:06,842 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 12:13:06,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 12:13:06,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:13:06,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 12:13:06,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 12:13:06,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 12:13:06,843 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 12:13:06,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-835465265] 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:06,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 12:13:06,843 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] 12:13:06,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] 12:13:06,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#640563460] 12:13:06,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:06,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 12:13:06,844 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:13:06,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 12:13:06,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:06,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 0 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:06,845 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:06,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 12:13:06,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:13:06,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015, primaryVersion=13] 12:13:06,848 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015 for shard member-2-shard-cars-oper_testAddShardReplica 12:13:06,848 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddShardReplica, persistent : true 12:13:06,848 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,849 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 12:13:06,849 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#1651276828] 12:13:06,849 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1433473529 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 12:13:06,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1433473529] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#1651276828] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,849 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddShardReplica with journal batch size 1 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 12:13:06,850 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 12:13:06,850 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 12:13:06,850 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 12:13:06,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 12:13:06,850 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 12:13:06,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 1 12:13:06,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:13:06,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddShardReplica 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@38eb3f46, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=null] 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddShardReplica: 229 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica), Chunk: 1/1 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 12:13:06,851 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 12:13:06,852 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 12:13:06,852 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:06,852 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 12:13:06,852 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:13:06,852 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 12:13:06,853 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddShardReplica to state 12:13:06,853 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:06,853 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 12:13:06,853 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,853 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@736dad9c, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 12:13:06,853 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to -1 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#86141822] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 12:13:06,854 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#86141822] 12:13:06,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 690.3 μs 12:13:06,855 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 12:13:06,855 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 12:13:06,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 12:13:06,855 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#86141822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,855 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 113.0 μs 12:13:06,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: SnapshotComplete received 12:13:06,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a snapshot response 12:13:06,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a journal response 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddShardReplica] 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d7bdce61-ba17-45fd-89fd-3a3e79199cc1}, logIndex: 0, isSendImmediate: true 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddShardReplica 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 12:13:06,856 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:06,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015 12:13:06,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 12:13:06,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 12:13:06,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:06,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:06,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 0 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:06,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 0 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 0 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 12:13:06,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 12:13:06,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 12:13:06,859 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 12:13:06,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 12:13:06,859 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,859 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 12:13:06,860 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:06,860 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:13:06,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#640563460] 12:13:06,860 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:06,860 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 12:13:06,860 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] 12:13:06,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] 12:13:06,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:13:06,883 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1285484365874183521] 12:13:06,884 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:06,887 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:06,887 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:06,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:06,892 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:06,901 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:06,903 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:06,906 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 12:13:06,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1464723992] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,912 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:06,913 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:06,917 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:06,919 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@377cdb5d 12:13:06,921 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 12:13:06,922 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:06,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:06,923 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:06,925 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 12:13:06,926 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:06,928 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:06,929 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:06,931 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1302155870] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:06,932 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:06,932 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:06,932 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,932 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:06,932 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:06,932 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:06,932 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,932 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:06,934 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:06,940 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:06,941 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5b3d15e3 12:13:06,941 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 12:13:06,942 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:06,986 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:06,986 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:06,986 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 12:13:06,986 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 12:13:06,986 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 12:13:06,986 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 12:13:06,986 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:06,987 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 12:13:07,153 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 0 12:13:07,153 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:07,153 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 0 12:13:07,153 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 12:13:07,153 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:07,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 12:13:07,154 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 12:13:07,162 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 12:13:07,473 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 12:13:07,482 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 12:13:07,753 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:07,753 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:07,754 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 12:13:07,754 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:07,754 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:07,754 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:07,754 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:07,754 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:07,754 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 12:13:08,6 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:08,6 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 12:13:08,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 12:13:08,6 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:08,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:08,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:08,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 12:13:08,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 12:13:08,44 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:13:08,45 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 12:13:08,45 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,45 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 12:13:08,47 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074, primaryVersion=13] 12:13:08,50 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074 for shard member-3-shard-cars-config_testAddShardReplica 12:13:08,51 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 12:13:08,51 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1494926776] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,52 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 12:13:08,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 12:13:08,52 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1310628036] 12:13:08,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:08,52 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1616635005 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 12:13:08,52 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 12:13:08,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 3 12:13:08,52 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1310628036] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,52 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1616635005] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:13:08,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-config_testAddShardReplica 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ae5787a, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 12:13:08,53 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testAddShardReplica: 229 12:13:08,53 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-3-shard-cars-config_testAddShardReplica), Chunk: 1/1 12:13:08,53 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:08,53 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 12:13:08,53 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 3 12:13:08,53 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 12:13:08,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 12:13:08,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 12:13:08,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 12:13:08,54 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 12:13:08,54 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 12:13:08,54 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 12:13:08,54 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 12:13:08,55 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:08,55 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,55 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:13:08,55 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 12:13:08,55 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddShardReplica to state 12:13:08,55 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:08,55 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 12:13:08,55 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2971a387, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 12:13:08,56 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1940353656] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 12:13:08,56 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1940353656] 12:13:08,56 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 12:13:08,56 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 326.0 μs 12:13:08,57 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 12:13:08,57 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 12:13:08,57 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 102.3 μs 12:13:08,57 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotComplete received 12:13:08,57 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a snapshot response 12:13:08,57 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1940353656] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,57 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a journal response 12:13:08,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testAddShardReplica] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=c11d9a46-7816-4aa5-aede-b20a98713338}, logIndex: 1, isSendImmediate: true 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testAddShardReplica 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testAddShardReplica 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 12:13:08,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:08,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 12:13:08,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074 12:13:08,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 12:13:08,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:08,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:08,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 12:13:08,59 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 12:13:08,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,59 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 12:13:08,59 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,60 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:08,60 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 12:13:08,60 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,60 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1464723992] 12:13:08,60 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:08,60 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1494926776] 12:13:08,60 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1494926776] 12:13:08,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 12:13:08,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 12:13:08,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 1 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:08,61 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 12:13:08,61 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 12:13:08,61 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 12:13:08,61 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 12:13:08,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1302155870] 12:13:08,61 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:13:08,61 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:08,61 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 12:13:08,61 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 12:13:08,62 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 12:13:08,62 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,62 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 12:13:08,62 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 12:13:08,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 12:13:08,62 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 12:13:08,63 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,63 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015, primaryVersion=13] 12:13:08,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015 for shard member-3-shard-cars-oper_testAddShardReplica 12:13:08,65 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-oper_testAddShardReplica, persistent : true 12:13:08,66 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1481708324] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#1100719696 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 12:13:08,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#1100719696] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,66 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#759380640] 12:13:08,66 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testAddShardReplica with journal batch size 1 12:13:08,66 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#759380640] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,67 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 12:13:08,67 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:08,67 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 12:13:08,67 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 3 12:13:08,67 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 12:13:08,67 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-oper_testAddShardReplica 12:13:08,67 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:08,67 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:08,68 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 12:13:08,68 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 12:13:08,68 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:08,68 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@501d130a, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,68 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 12:13:08,68 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 12:13:08,68 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testAddShardReplica: 229 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-3-shard-cars-oper_testAddShardReplica), Chunk: 1/1 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 3 12:13:08,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 12:13:08,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 12:13:08,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 12:13:08,69 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 12:13:08,69 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:08,69 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,69 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 12:13:08,69 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddShardReplica to state 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:08,70 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 12:13:08,70 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@27e667b9, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 12:13:08,70 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:13:08,71 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1540300053] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 12:13:08,71 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 339.2 μs 12:13:08,71 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 12:13:08,71 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 12:13:08,71 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1540300053] 12:13:08,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1540300053] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,72 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: SnapshotComplete received 12:13:08,72 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a journal response 12:13:08,72 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a snapshot response 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testAddShardReplica] 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8cc368de-9b36-4670-977d-cb2bbe6090f2}, logIndex: 1, isSendImmediate: true 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testAddShardReplica 12:13:08,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 12:13:08,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:08,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testAddShardReplica 12:13:08,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 12:13:08,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 12:13:08,74 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,74 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 12:13:08,74 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 329.4 μs 12:13:08,74 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,75 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:08,75 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015 12:13:08,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 1 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:13:08,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 12:13:08,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 12:13:08,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 12:13:08,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 12:13:08,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 12:13:08,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 12:13:08,76 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 12:13:08,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 12:13:08,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 12:13:08,76 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1302155870] 12:13:08,76 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 12:13:08,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:13:08,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 12:13:08,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:08,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 12:13:08,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 12:13:08,77 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 12:13:08,77 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:08,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 12:13:08,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 12:13:08,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 12:13:08,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 12:13:08,77 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 12:13:08,77 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 12:13:08,77 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:08,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:08,78 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1481708324] 12:13:08,78 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1481708324] 12:13:08,78 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-835465265] 12:13:08,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:08,78 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 12:13:08,78 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 12:13:08,78 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] 12:13:08,78 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] 12:13:08,78 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#640563460] 12:13:08,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:08,78 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] 12:13:08,78 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] 12:13:08,121 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-835465265] 12:13:08,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 12:13:08,122 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 12:13:08,136 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 12:13:08,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=45991, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:08,158 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 12:13:08,160 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 12:13:08,184 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 25.17 ms 12:13:08,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 12:13:08,210 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 12:13:08,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 12:13:08,214 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,214 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 12:13:08,214 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 12:13:08,214 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 12:13:08,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 12:13:08,219 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 12:13:08,222 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 12:13:08,222 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 12:13:08,222 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 12:13:08,222 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 12:13:08,222 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 12:13:08,232 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,239 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 12:13:08,239 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,239 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,239 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 12:13:08,239 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:13:08,239 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testAddShardReplica 12:13:08,241 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 12:13:08,245 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:13:08,245 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testAddShardReplica 12:13:08,245 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 12:13:08,245 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 12:13:08,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 12:13:08,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,247 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,247 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 12:13:08,248 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 12:13:08,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 12:13:08,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 12:13:08,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 2 12:13:08,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 12:13:08,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@77939444}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,252 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,253 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 12:13:08,253 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 12:13:08,254 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#860433332] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 12:13:08,254 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#860433332] 12:13:08,254 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 12:13:08,254 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#860433332] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,254 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 106.6 μs 12:13:08,263 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1464723992] 12:13:08,263 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,263 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 12:13:08,263 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 12:13:08,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 12:13:08,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:13:08,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 12:13:08,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:13:08,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 12:13:08,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 12:13:08,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=9620, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:08,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 12:13:08,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:13:08,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 12:13:08,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 12:13:08,271 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,271 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,273 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,273 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,273 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 12:13:08,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@28df01a4}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,274 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 12:13:08,274 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 101.8 μs 12:13:08,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 12:13:08,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 12:13:08,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 12:13:08,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:13:08,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 12:13:08,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 12:13:08,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,275 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 12:13:08,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,276 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 12:13:08,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1ce73aae}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 12:13:08,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 12:13:08,276 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 707.5 μs 12:13:08,277 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 12:13:08,277 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 76.84 μs 12:13:08,279 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testAddShardReplica 12:13:08,279 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 12:13:08,279 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-config_testAddShardReplica 12:13:08,279 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 12:13:08,279 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 12:13:08,280 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 12:13:08,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,281 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,281 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 12:13:08,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 12:13:08,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 12:13:08,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 3 12:13:08,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 12:13:08,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007fe3f494d658@63171c62 12:13:08,283 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,284 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,284 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,284 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:08,284 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:08,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 12:13:08,286 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 12:13:08,286 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:08,286 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 12:13:08,287 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1302155870] 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 12:13:08,288 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:08,288 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 12:13:08,289 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 12:13:08,289 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=9980, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:08,289 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 12:13:08,290 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 12:13:08,290 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 583.6 μs 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 12:13:08,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testAddShardReplica 12:13:08,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 12:13:08,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 12:13:08,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testAddShardReplica 12:13:08,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 12:13:08,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 12:13:08,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,294 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,295 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 2 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@14684827}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 12:13:08,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 12:13:08,296 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-581046243] 12:13:08,296 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-581046243] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 12:13:08,296 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 12:13:08,296 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-581046243] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,296 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 97.67 μs 12:13:08,297 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#640563460] 12:13:08,297 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,297 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 12:13:08,297 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 12:13:08,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 12:13:08,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:13:08,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 12:13:08,298 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 12:13:08,298 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 12:13:08,298 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 12:13:08,298 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:13:08,298 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 12:13:08,298 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,298 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 12:13:08,298 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 12:13:08,299 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 12:13:08,299 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 12:13:08,299 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,299 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,299 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 12:13:08,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3f87797b}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,300 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 12:13:08,300 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 78.12 μs 12:13:08,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 12:13:08,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 12:13:08,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@220a1f53}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,300 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 12:13:08,300 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 60.63 μs 12:13:08,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 12:13:08,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 12:13:08,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 12:13:08,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 12:13:08,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=12191, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:08,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,302 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 12:13:08,302 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 12:13:08,302 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 335.3 μs 12:13:08,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,306 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1553541589] 12:13:08,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:08,307 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074] 12:13:08,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074] 12:13:08,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1106890079] 12:13:08,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:08,307 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,358 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,408 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,459 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,560 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,611 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,661 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,712 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testAddShardReplica 12:13:08,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 12:13:08,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-oper_testAddShardReplica 12:13:08,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 12:13:08,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 12:13:08,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:13:08,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 12:13:08,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 12:13:08,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:08,763 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 3 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007fe3f494d658@740dbd2b 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:08,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:08,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 12:13:08,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:08,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:08,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 12:13:08,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 12:13:08,771 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:08,773 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:08,774 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 12:13:08,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 12:13:08,774 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:08,773 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:08,774 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:08,774 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:08,774 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:08,814 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] 12:13:08,815 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-835465265] 12:13:08,815 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:08,815 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] 12:13:08,815 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-932746685] 12:13:08,815 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#640563460] 12:13:08,815 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:08,815 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] 12:13:08,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#596724766] 12:13:08,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1464723992] 12:13:08,816 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:08,816 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1494926776] 12:13:08,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1494926776] 12:13:08,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1302155870] 12:13:08,816 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:08,816 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1481708324] 12:13:08,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1481708324] 12:13:08,817 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:08,817 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 12:13:08,817 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 12:13:08,817 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 12:13:08,818 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 12:13:08,818 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 12:13:08,818 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:08,818 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:08,818 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:08,819 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node [akka://cluster-test@127.0.0.1:2557] as [Down] 12:13:08,819 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 12:13:08,819 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 12:13:08,819 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onShutDown 12:13:08,820 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testAddShardReplica 12:13:08,820 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 12:13:08,820 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:08,820 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:08,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:08,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:08,823 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [akka://cluster-test@127.0.0.1:2558] 12:13:08,823 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 12:13:08,824 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Shutting down... 12:13:08,824 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Successfully shut down 12:13:08,831 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:08,831 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:08,831 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:08,831 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:08,833 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:08,849 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:08,862 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2947094820489326859] 12:13:08,862 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:08,865 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:08,865 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:08,866 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:08,867 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:08,877 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:08,878 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:08,880 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 12:13:08,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#2043563568] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,891 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:08,891 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onSnapshotOffer: ShardManagerSnapshot [ShardList = [cars] ] 12:13:08,891 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 12:13:08,892 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:08,892 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 12:13:08,896 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@77cc525 12:13:08,897 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 12:13:08,897 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#2043563568] 12:13:08,897 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:08,897 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 12:13:08,897 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:08,897 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 12:13:08,898 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 12:13:08,898 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 12:13:08,899 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1674859810] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,899 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#326681403] 12:13:08,899 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#2059577961] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,899 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#326681403] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#11943676] 12:13:08,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotOffer called. 12:13:08,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 12:13:08,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 12:13:08,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 12:13:08,902 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#2059577961 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 12:13:08,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 12:13:08,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-2145613403] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 12:13:08,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-2145613403] 12:13:08,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 3.087 ms 12:13:08,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 12:13:08,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 12:13:08,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 92.25 μs 12:13:08,911 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-2145613403] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:08,911 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 11.14 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 1, size: 267 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 1 to 1 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 2, size: 235 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 2 to 2 12:13:08,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 12:13:08,920 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4df7c8c1}, purged=MutableUnsignedLongSet{size=0}} 12:13:08,920 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 3, size: 56 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 3 to 3 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 21.13 ms - Switching actor to Follower - last log index = 3, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 3 12:13:08,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:08,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:08,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 12:13:08,921 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 12:13:08,921 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 12:13:08,921 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1674859810] 12:13:08,921 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1674859810] 12:13:08,922 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#2043563568] 12:13:08,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:08,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 12:13:08,922 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 12:13:08,922 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 12:13:08,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 12:13:09,792 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:09,840 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [1285484365874183521] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:09,842 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [1285484365874183521] but current is [-2947094820489326859]. Reason: ActorSystem terminated 12:13:09,939 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:13:09,940 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:13:09,956 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:09,956 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:09,956 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:09,956 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 12:13:09,956 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = null 12:13:09,956 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 12:13:09,956 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 12:13:09,956 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 12:13:09,957 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 12:13:09,957 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 12:13:10,11 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#179510389]] intended for system uid [1285484365874183521], self uid is [-2947094820489326859] 12:13:10,43 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:10,354 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:10,354 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:10,363 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1081116015] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:10,900 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:13:11,793 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-153720025]] intended for system uid [1285484365874183521], self uid is [-2947094820489326859] 12:13:11,919 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:13:12,32 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#179510389]] intended for system uid [1285484365874183521], self uid is [-2947094820489326859] 12:13:12,939 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:13:13,873 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:13,874 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:13,875 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Down)]. 12:13:13,959 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 12:13:14,52 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#179510389]] intended for system uid [1285484365874183521], self uid is [-2947094820489326859] 12:13:14,121 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Down)]. 12:13:14,121 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:14,121 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:14,892 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2557] 12:13:14,893 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [1285484365874183521] but current is [-2947094820489326859]. Reason: Cluster member removed, previous status [Down] 12:13:14,893 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:14,893 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:14,897 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:14,897 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:14,898 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [1285484365874183521] but current is [-2947094820489326859]. Reason: Cluster member removed, previous status [Down] 12:13:14,979 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:14,982 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 12:13:14,982 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:14,982 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:14,983 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 12:13:14,983 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:14,983 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 12:13:14,983 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 12:13:14,984 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 12:13:14,987 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6708095589, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 12:13:14,990 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=16630, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 12:13:14,992 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 12:13:14,992 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 12:13:14,992 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1297976074], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 370.2 μs 12:13:14,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 12:13:14,995 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 12:13:14,996 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 12:13:14,996 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 12:13:14,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 12:13:14,996 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 12:13:14,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 12:13:14,996 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:14,996 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:14,998 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:14,998 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:15,3 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:15,4 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:15,4 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:15,4 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:15,6 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:15,18 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:15,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 12:13:15,18 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 12:13:15,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onShutDown 12:13:15,19 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddShardReplica 12:13:15,19 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 12:13:15,19 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:15,19 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:15,19 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:15,19 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 12:13:15,20 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 12:13:15,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onShutDown 12:13:15,20 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddShardReplica 12:13:15,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 12:13:15,20 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:15,20 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:15,20 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:15,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:15,911 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:15,912 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:15,912 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:15,912 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 12:13:15,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 12:13:15,913 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:15,913 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 12:13:15,913 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 12:13:15,913 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 12:13:15,913 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 12:13:15,913 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 12:13:16,11 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-2947094820489326859] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:16,42 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:16,42 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:16,42 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-992355537910435549] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:16,43 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:16.046] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:16,51 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 12:13:16,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 12:13:16,51 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 12:13:16,51 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onShutDown 12:13:16,51 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating leader transfer 12:13:16,52 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 12:13:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c2b6941 12:13:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c2b6941 12:13:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 12:13:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Attempting to transfer leadership 12:13:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:16,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:16,932 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:21,12 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 12:13:21,13 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:21,13 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:22,518 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:13:23,52 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 12:13:23,52 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:23,53 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:23,798 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:13:24,72 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:25,341 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 12:13:25,341 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.290 s 12:13:25,341 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 12:13:25,344 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: leader transfer failed - sending PoisonPill 12:13:25,344 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Leadership transfer expired 12:13:25,345 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplica 12:13:25,345 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 12:13:25,345 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:25,346 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:25,346 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 12:13:25,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 12:13:25,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onShutDown 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating leader transfer 12:13:25,347 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@16162349 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@16162349 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Attempting to transfer leadership 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:25,347 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testAddShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica] 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 965.3 ms 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 12:13:26,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: leader transfer failed - sending PoisonPill 12:13:26,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:26,313 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 12:13:26,313 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 12:13:26,313 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplica 12:13:26,313 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 12:13:26,314 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:26,314 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:26,315 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:26,315 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:26,319 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:26.321] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:26,347 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:26,357 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4509796894591301643] 12:13:26,358 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:26,360 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:26,360 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:26,360 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:26,360 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:26,372 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,373 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:26,373 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:26,373 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:26,375 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,378 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,384 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:26,384 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1720727980] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,388 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:26,388 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,389 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:26,389 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,389 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,393 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6e7112c0 12:13:26,394 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 12:13:26,395 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:26,395 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,395 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,395 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,395 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,395 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1338645336] 12:13:26,395 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,396 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,396 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,396 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1882001401] 12:13:26,396 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,396 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,396 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1338645336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,397 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-84313926 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,397 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-84313926] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,397 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#87359504] 12:13:26,397 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,397 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#87359504] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,398 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,398 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,399 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1281793669 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,399 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1882001401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,399 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1281793669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,399 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#623672156] 12:13:26,399 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#623672156] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,399 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,400 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,400 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1797745314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,400 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:26,400 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,400 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,401 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:26,400 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,401 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:26,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,401 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,401 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,401 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,402 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:26,402 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,402 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,402 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:26,402 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,402 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,402 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,402 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,402 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,402 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,402 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,402 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,403 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,404 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5825bd16 12:13:26,404 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 12:13:26,404 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:26,404 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-254490184] 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#356806672] 12:13:26,405 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,405 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-254490184] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,406 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-863920824] 12:13:26,406 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1584264968 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,406 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1584264968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,406 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,407 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,415 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,415 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:26,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,415 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,415 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,416 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:13:26,416 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-896659077] 12:13:26,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-184952356 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,417 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 12:13:26,418 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 12:13:26,418 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 12:13:26,418 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,420 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,420 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:26,420 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,421 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,421 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,421 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3441908962155203782] 12:13:26,427 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:26,429 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:26,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:26,430 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:26,430 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:26,442 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,443 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,445 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,452 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1755465618] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,452 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:26,457 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:26,457 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,457 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,463 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 12:13:26,464 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,466 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5dc5d8a8 12:13:26,466 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:26,466 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,466 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,467 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,468 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,469 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,469 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,469 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,469 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,471 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:26,472 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#486825075 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,472 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#1617056106] 12:13:26,472 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1428150906 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,472 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-798383481] 12:13:26,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1242716858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#857967914] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1754097498] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#486825075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#1617056106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1428150906] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,476 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-798383481] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,477 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,478 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 12:13:26,478 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:26,478 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@33b82de4 12:13:26,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:26,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,481 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427703606] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,482 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,482 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,482 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1932451142 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,483 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1932451142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,483 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#911575169] 12:13:26,483 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,484 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:13:26,484 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,485 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,485 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,485 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,486 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:26,486 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,485 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,487 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,487 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,487 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,490 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:26,492 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,493 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,493 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,493 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,493 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,497 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:26,501 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:26,501 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,501 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,501 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,501 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,501 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,504 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,504 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,504 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:26,504 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,504 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:26,504 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-1654183960] 12:13:26,505 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1185543994 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,505 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,506 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,506 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:13:26,506 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,506 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,506 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,506 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,506 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 12:13:26,521 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3477081586860934571] 12:13:26,521 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:26,525 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:26,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:26,528 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:26,528 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:26,536 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,539 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,541 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,548 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1293901749] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,551 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:26,552 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:26,552 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,552 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,555 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,556 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@28c095d1 12:13:26,556 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 12:13:26,556 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,558 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:26,559 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:26,559 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,560 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,563 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,563 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1668181019] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,564 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,564 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,565 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,565 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1701611518 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1701611518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1915410575] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1186627955] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1626410726 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1626410726] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#676009527] 12:13:26,572 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#676009527] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,573 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1837998588] 12:13:26,573 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1837998588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,575 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:26,575 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,575 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,575 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,575 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,575 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:13:26,575 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,577 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:26,577 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,577 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,577 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:26,577 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,579 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:26,582 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,583 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 12:13:26,584 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1828f80d 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:26,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,585 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,585 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,586 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,586 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,586 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-532372963 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,587 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1346968176] 12:13:26,587 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#227266050] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,587 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-532372963] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:26,587 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,587 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,587 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,587 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,590 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#227266050] 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,590 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1783367638] 12:13:26,591 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 12:13:26,591 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 12:13:26,591 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#127608683 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,591 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-253963877] 12:13:26,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,592 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,592 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,592 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:13:26,592 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,593 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,593 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,593 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,593 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:26,593 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:26,593 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:26,593 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 12:13:26,593 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,593 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,593 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 12:13:26,593 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,593 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 12:13:26,594 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 12:13:26,720 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,721 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 12:13:26,721 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,771 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 12:13:26,771 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,790 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 12:13:26,790 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 12:13:26,820 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:26,820 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 12:13:26,820 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 12:13:26,820 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:26,820 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 12:13:26,820 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:26,821 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 12:13:26,821 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 12:13:26,821 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 12:13:26,821 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 12:13:27,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,101 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 12:13:27,150 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,150 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,150 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 12:13:27,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,181 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 12:13:27,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,200 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 12:13:27,372 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:27,372 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:27,372 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:27,372 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:27,372 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,372 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:27,372 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,373 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,380 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:27,380 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:27,380 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:27,381 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,381 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:27,464 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,465 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,520 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,520 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,520 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 12:13:27,520 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 12:13:27,520 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,521 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,522 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,522 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,522 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,522 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,522 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,522 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,523 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,523 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,523 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,523 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,523 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,523 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,523 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,523 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,523 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,522 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,523 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,523 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,523 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,523 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,523 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:27,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:27,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 12:13:27,524 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:27,524 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:27,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 12:13:27,525 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,525 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,525 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,525 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 12:13:27,525 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 12:13:27,525 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 12:13:27,525 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 12:13:27,525 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 12:13:27,525 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,526 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,526 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,526 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,526 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,526 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,526 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,526 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 12:13:27,526 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,527 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,527 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,527 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,527 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,528 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,528 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,528 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,528 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 12:13:27,528 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,542 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:13:27,590 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,590 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,590 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 12:13:27,590 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,590 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,592 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:13:27,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 12:13:27,595 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 12:13:27,595 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,595 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 12:13:27,595 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,596 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,596 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,596 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,596 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,596 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 12:13:27,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,600 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 12:13:27,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,601 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,601 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:27,601 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,601 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,601 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 12:13:27,601 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,601 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:27,601 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,601 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:27,601 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 12:13:27,601 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,602 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,602 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:27,602 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,602 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,602 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 12:13:27,603 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 12:13:27,603 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 12:13:27,603 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 12:13:27,603 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,603 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 12:13:27,603 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,604 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,604 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,604 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 12:13:27,604 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,604 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,604 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 12:13:27,604 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1720727980] 12:13:27,604 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 12:13:27,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:27,604 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1338645336] 12:13:27,604 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 12:13:27,604 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1338645336] 12:13:27,604 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 12:13:27,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,605 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1755465618] 12:13:27,605 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:27,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 12:13:27,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:27,605 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1242716858] 12:13:27,605 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1242716858] 12:13:27,606 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1293901749] 12:13:27,606 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:27,606 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1668181019] 12:13:27,606 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1668181019] 12:13:27,609 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 12:13:27,609 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 12:13:27,610 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 12:13:27,610 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 12:13:27,610 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 12:13:27,611 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 12:13:27,611 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:27,611 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:13:27,611 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 12:13:27,611 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:27,611 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:13:27,612 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,612 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,612 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,612 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,612 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,612 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,612 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,612 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,612 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards)] 12:13:27,612 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,612 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards)] 12:13:27,612 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards)] 12:13:27,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:27,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 12:13:27,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 12:13:27,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3a4b235a-7c33-4270-9d65-7bd15d8128b8}, logIndex: 0, isSendImmediate: true 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=6a2d36ad-6884-479e-b56e-b19b3efd113d}, logIndex: 0, isSendImmediate: true 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=5a9c1f47-6d17-4733-826f-b6715bb49047}, logIndex: 0, isSendImmediate: true 12:13:27,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 12:13:27,614 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:27,615 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,615 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,615 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,615 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,616 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,616 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 12:13:27,616 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,616 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,616 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,616 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,616 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 12:13:27,617 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:13:27,617 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 12:13:27,617 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,617 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:13:27,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,617 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,618 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,618 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,618 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,619 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,620 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,620 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,620 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards)] 12:13:27,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,621 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,621 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,621 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,621 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,621 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,621 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,621 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,622 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f8a7fd17-af2f-4447-99ef-467d1b4f4dce}, logIndex: 0, isSendImmediate: true 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:27,623 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,624 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 12:13:27,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:13:27,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 12:13:27,624 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,625 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 12:13:27,625 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 12:13:27,625 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 12:13:27,626 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1720727980] 12:13:27,626 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,626 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,626 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:27,626 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1338645336] 12:13:27,626 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,626 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 12:13:27,626 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 12:13:27,626 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 12:13:27,626 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1338645336] 12:13:27,626 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 12:13:27,626 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:27,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1720727980] 12:13:27,627 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:13:27,627 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1882001401] 12:13:27,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1882001401] 12:13:27,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1797745314] 12:13:27,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:27,628 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-254490184] 12:13:27,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-254490184] 12:13:27,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1797745314] 12:13:27,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:13:27,628 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#356806672] 12:13:27,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#356806672] 12:13:27,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1755465618] 12:13:27,629 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:27,629 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1242716858] 12:13:27,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1242716858] 12:13:27,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1755465618] 12:13:27,629 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:13:27,629 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#857967914] 12:13:27,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#857967914] 12:13:27,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1754097498] 12:13:27,630 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:27,630 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427703606] 12:13:27,630 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427703606] 12:13:27,630 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1754097498] 12:13:27,630 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 12:13:27,630 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-736292151] 12:13:27,630 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-736292151] 12:13:27,630 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1293901749] 12:13:27,630 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:27,631 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1668181019] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1668181019] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1293901749] 12:13:27,631 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:13:27,631 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1915410575] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1915410575] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1186627955] 12:13:27,631 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:27,631 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#227266050] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#227266050] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1186627955] 12:13:27,631 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:13:27,631 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1783367638] 12:13:27,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#1783367638] 12:13:27,631 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,632 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 12:13:27,632 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,632 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,632 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,632 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,632 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,633 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:27,633 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:13:27,633 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,633 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,633 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,634 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,634 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:27,634 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:27,635 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:27,635 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:27,639 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:27,639 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:27,639 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:27,639 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:27,642 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:27,652 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,653 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:13:27,653 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,654 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:27,654 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:27,654 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,654 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,654 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:27,654 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:27,655 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:27,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,655 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:13:27,656 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:27,656 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:27,656 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:27,656 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:27,656 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:27,656 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:27,657 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:27,657 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:28,391 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:28,651 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-3477081586860934571] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:28,671 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3441908962155203782] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:28,680 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:28,681 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:28,682 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:28.684] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:28,688 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:28,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:28,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:28,688 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:13:28,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:28,688 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 12:13:28,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@747a79f9 12:13:28,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@747a79f9 12:13:28,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 12:13:28,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 12:13:28,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 12:13:28,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:28,689 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c81bb4d 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c81bb4d 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 12:13:28,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 12:13:28,690 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:28,690 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:29,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 12:13:29,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 12:13:29,121 PM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 432.2 ms 12:13:29,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 12:13:29,121 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 431.9 ms 12:13:29,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 12:13:29,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 12:13:29,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 12:13:29,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 12:13:29,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 12:13:29,122 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 12:13:29,122 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 12:13:29,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:29,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:29,122 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:29,122 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:29,123 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 12:13:29,123 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:29,123 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:29,123 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:29,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:29,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 12:13:29,123 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 12:13:29,123 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73742a0b 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73742a0b 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:29,124 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50fc6d67 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:29,124 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50fc6d67 12:13:29,124 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 12:13:29,124 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 12:13:29,124 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 12:13:29,124 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:29,124 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 12:13:29,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 12:13:29,511 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 387.3 ms 12:13:29,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 12:13:29,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 12:13:29,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 12:13:29,512 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 12:13:29,512 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:29,521 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 12:13:29,521 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 397.7 ms 12:13:29,521 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 12:13:29,521 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 12:13:29,521 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 12:13:29,522 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 12:13:29,522 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 12:13:29,522 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:29,522 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:29,523 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:29,523 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:29,526 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:29.528] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:29,551 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:29,558 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2655746057480066328] 12:13:29,559 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:29,560 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:29,560 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:29,561 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:29,561 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:29,571 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,573 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,573 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:29,573 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:29,573 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:29,574 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 12:13:29,577 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1168530513] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,579 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:29,581 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:29,581 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 12:13:29,581 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,581 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 12:13:29,584 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 12:13:29,585 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6433b157 12:13:29,585 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 12:13:29,585 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,587 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,587 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 12:13:29,587 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 12:13:29,587 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 12:13:29,587 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1348655304] 12:13:29,588 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-default-config_testGetPrefixShardRole, persistent : true 12:13:29,588 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 12:13:29,589 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-default-config_testGetPrefixShardRole with journal batch size 1 12:13:29,589 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#1544307157] 12:13:29,590 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:29,592 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1348655304] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,592 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1624700168] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,592 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#1544307157] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,593 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 12:13:29,595 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-1396904282 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 12:13:29,595 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-1396904282] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,596 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,596 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:29,596 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:29,596 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,596 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 12:13:29,596 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,596 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 12:13:29,596 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 12:13:29,596 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,596 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 12:13:29,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 12:13:29,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 12:13:29,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-config_testGetPrefixShardRole 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@111bcf56 12:13:29,597 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 12:13:29,597 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:29,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,597 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-339538082] 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Received ElectionTimeout 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 12:13:29,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 12:13:29,597 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-339538082] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,597 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 12:13:29,598 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole 12:13:29,598 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config_testGetPrefixShardRole, leaderId=member-1-shard-default-config_testGetPrefixShardRole, leaderPayloadVersion=13] 12:13:29,598 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-default-oper_testGetPrefixShardRole, persistent : true 12:13:29,598 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 12:13:29,598 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 12:13:29,598 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-700040855 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,598 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-700040855] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,598 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#929026842] 12:13:29,598 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#929026842] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,599 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-default-oper_testGetPrefixShardRole with journal batch size 1 12:13:29,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,599 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:29,599 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 12:13:29,599 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 12:13:29,600 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 12:13:29,600 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Received ElectionTimeout 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 12:13:29,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 12:13:29,600 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 12:13:29,600 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 12:13:29,601 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-oper_testGetPrefixShardRole, leaderId=member-1-shard-default-oper_testGetPrefixShardRole, leaderPayloadVersion=13] 12:13:29,601 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 12:13:29,601 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 12:13:29,646 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1168530513] 12:13:29,647 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 12:13:29,647 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1348655304] 12:13:29,647 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 12:13:29,647 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 12:13:29,647 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 12:13:29,647 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onShutDown 12:13:29,647 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@22831d8e 12:13:29,647 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@22831d8e 12:13:29,648 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-default-config_testGetPrefixShardRole 12:13:29,648 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Aborting 0 pending queued transactions 12:13:29,648 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:29,648 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:29,648 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 12:13:29,648 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,648 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 12:13:29,649 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onShutDown 12:13:29,649 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7a11e493 12:13:29,649 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7a11e493 12:13:29,649 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1168530513] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1168530513] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,649 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-default-oper_testGetPrefixShardRole 12:13:29,649 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Aborting 0 pending queued transactions 12:13:29,649 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:29,649 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1168530513] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1168530513] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,650 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:29,651 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:29,651 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:29,653 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:29,668 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:29,676 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6977667437913959912] 12:13:29,676 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:29,681 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:29,681 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:29,685 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:29,685 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:29,696 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,697 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:29,697 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:29,698 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:29,700 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,702 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,706 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:29,707 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-745327759] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,711 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:29,712 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,712 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,712 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,712 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,712 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,714 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:29,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@360292cb 12:13:29,715 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 12:13:29,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,715 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,715 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:29,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,716 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,716 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,716 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,717 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1486365940] 12:13:29,717 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1341874809] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,717 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,718 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,718 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1938607836 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,718 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#674702592] 12:13:29,718 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,718 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,719 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1341874809 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,719 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1486365940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,719 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-725590940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,719 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1938607836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,719 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#674702592] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,720 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,720 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,720 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-725590940] 12:13:29,720 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-292009172] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,723 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:29,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,723 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,723 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,723 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,724 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,724 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:29,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:29,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,725 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:29,725 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,725 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,725 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:29,725 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,725 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,726 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,726 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,727 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:29,727 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7680cbb1 12:13:29,728 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 12:13:29,728 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,728 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,732 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 12:13:29,733 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,733 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,733 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1797050231] 12:13:29,733 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,734 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,734 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,734 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-836004353] 12:13:29,734 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,734 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,734 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1784696230 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,734 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,734 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1797050231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,735 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-836004353] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,735 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,736 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:29,736 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,739 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1691003594] 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,740 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:29,740 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,740 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,740 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,741 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,741 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,743 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#850952066] 12:13:29,743 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#977621040 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,743 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,743 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,743 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,754 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7454115254492104678] 12:13:29,754 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:29,763 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:29,763 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:29,763 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:29,763 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:29,767 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,768 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,770 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,776 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#28946889] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,782 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:29,784 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:29,784 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,784 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,786 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:29,787 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@fedc32e 12:13:29,787 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 12:13:29,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,788 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,788 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,789 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,790 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,790 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#744769094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,790 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-412799205] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,790 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,790 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,791 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,793 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1161143964 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,793 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1666984176] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,793 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1742702221] 12:13:29,794 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-1927265341] 12:13:29,794 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:29,794 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1666984176 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,794 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,794 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,795 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,795 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:29,795 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,795 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,795 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,796 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:29,796 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-888637441] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1161143964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1742702221] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-1927265341] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,797 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,797 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:29,797 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,797 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,797 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:13:29,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,797 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,797 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,797 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,797 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:29,798 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 12:13:29,798 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,799 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@96fbd0b 12:13:29,803 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,803 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,803 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,804 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,804 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#886836659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,804 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2146393864 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,804 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1326503452] 12:13:29,804 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,804 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,805 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-175918158 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-577323184] 12:13:29,805 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,805 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2146393864] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,806 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,806 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,807 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,807 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,807 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:29,807 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,807 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:13:29,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,807 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,807 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,807 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,823 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:29,831 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:29,833 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:29,833 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,833 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,833 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,834 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,834 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,844 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5473855825911259761] 12:13:29,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:29,846 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:29,846 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:29,847 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:29,851 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:29,856 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,858 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,860 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,866 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1247225485] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,866 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:29,870 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:29,870 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,870 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,871 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:29,872 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7f1811ef 12:13:29,872 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 12:13:29,872 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,873 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:29,875 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,875 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,875 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,875 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,876 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,876 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,876 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,877 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,878 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#778755669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,878 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-993846346] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,878 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#745910747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,886 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:29,887 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-210460255 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,887 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1094826405 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,887 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#518173658] 12:13:29,887 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-210460255] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,889 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1514625770] 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,890 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,892 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:29,892 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,892 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:29,893 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:29,893 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,894 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,895 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1094826405] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,895 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#518173658] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,895 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1514625770] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,897 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:29,897 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,897 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,897 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,897 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,897 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,897 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:13:29,897 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,897 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,897 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,897 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,897 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,897 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,897 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,897 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,899 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,899 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:29,899 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4f1fe74e 12:13:29,899 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 12:13:29,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:29,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,905 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,905 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,905 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,905 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:29,906 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1160841099] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,906 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,906 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 12:13:29,906 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-907048627] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:29,907 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1478539224 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,907 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-453845510] 12:13:29,907 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,907 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-149221578 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,908 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#840807987] 12:13:29,908 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 12:13:29,908 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,908 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,908 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,909 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:13:29,909 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,909 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,909 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,909 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:29,909 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:29,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:29,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 12:13:29,910 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:29,910 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:29,910 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 12:13:29,910 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,910 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:29,910 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 12:13:30,697 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:30,697 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,697 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,698 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-745327759] 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:30,706 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,706 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,707 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-745327759] 12:13:30,707 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:30,707 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,707 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,707 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 12:13:30,707 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,707 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,707 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:13:30,707 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:13:30,707 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:13:30,707 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:30,708 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:13:30,708 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:13:30,708 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:30,708 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:13:30,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,708 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,708 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 12:13:30,708 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:30,708 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 12:13:30,708 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,708 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:30,709 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:30,709 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,709 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,710 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,710 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 12:13:30,710 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 12:13:30,710 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 12:13:30,710 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,712 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,712 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,715 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 12:13:30,715 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 12:13:30,715 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:13:30,715 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:13:30,715 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,715 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,715 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=4c9d998f-4dee-4d22-adf9-eca379c36a61}, logIndex: 1, isSendImmediate: true 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader elected 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=b3a5923e-7ced-4969-b813-ffffa6d38e4d}, logIndex: 1, isSendImmediate: true 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 12:13:30,716 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:13:30,716 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader elected 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 12:13:30,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=13a496e4-d361-4034-9d87-971ca2bf10b3}, logIndex: 1, isSendImmediate: true 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 12:13:30,717 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,717 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,717 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 12:13:30,717 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,717 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,717 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 12:13:30,717 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 12:13:30,717 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,717 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,717 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,717 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:30,718 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,718 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,718 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=57975bcd-01a0-41b5-99e8-7ad6a0c65d30}, logIndex: 1, isSendImmediate: true 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 12:13:30,718 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 12:13:30,718 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,718 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,718 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:13:30,718 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 12:13:30,718 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-745327759] 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,718 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,719 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,719 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,719 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,719 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,719 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,719 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,719 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,719 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,719 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,719 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,719 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 12:13:30,719 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,719 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,719 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 12:13:30,719 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,721 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,721 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,721 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,721 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,721 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,722 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,722 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,722 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 12:13:30,722 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 12:13:30,723 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 12:13:30,723 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,723 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,724 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,725 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,725 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,725 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,725 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,725 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,725 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,726 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,726 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,726 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,727 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,727 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,727 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,727 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 12:13:30,727 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,727 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,727 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,728 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,725 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,729 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,729 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 12:13:30,729 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 12:13:30,728 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,729 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,729 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,729 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,727 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 12:13:30,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 2, lastIndex: 2 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,730 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,730 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 12:13:30,730 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 12:13:30,729 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 2, lastIndex: 2 12:13:30,728 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,730 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,730 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,730 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,731 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,731 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,731 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,728 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:30,731 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,731 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,732 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 2, lastIndex: 2 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,731 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,729 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,729 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,732 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,732 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,732 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,732 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:13:30,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,732 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,732 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,732 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 12:13:30,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,732 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 12:13:30,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 12:13:30,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,733 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,732 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,733 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,733 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,733 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,733 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 12:13:30,733 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,733 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-745327759] 12:13:30,734 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:13:30,734 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-725590940] 12:13:30,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-725590940] 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 2, lastIndex: 2 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 12:13:30,734 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 12:13:30,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,734 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 12:13:30,734 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 12:13:30,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-292009172] 12:13:30,734 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:30,734 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1797050231] 12:13:30,734 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,734 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1797050231] 12:13:30,735 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 12:13:30,735 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,735 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,735 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,735 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,736 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,736 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:30,736 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 12:13:30,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-292009172] 12:13:30,737 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:13:30,737 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-836004353] 12:13:30,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-836004353] 12:13:30,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#28946889] 12:13:30,737 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:30,737 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#744769094] 12:13:30,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#744769094] 12:13:30,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#28946889] 12:13:30,738 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:13:30,738 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-412799205] 12:13:30,738 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-412799205] 12:13:30,738 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-888637441] 12:13:30,738 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:30,738 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1529835732] 12:13:30,738 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1529835732] 12:13:30,738 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-888637441] 12:13:30,738 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 12:13:30,738 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#886836659] 12:13:30,738 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#886836659] 12:13:30,738 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1247225485] 12:13:30,738 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:30,739 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#778755669] 12:13:30,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#778755669] 12:13:30,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1247225485] 12:13:30,739 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:13:30,739 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-993846346] 12:13:30,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-993846346] 12:13:30,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#745910747] 12:13:30,739 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:30,739 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1160841099] 12:13:30,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1160841099] 12:13:30,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#745910747] 12:13:30,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:13:30,740 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-907048627] 12:13:30,740 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-907048627] 12:13:30,740 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-745327759] 12:13:30,740 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:30,740 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,740 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-294027138] 12:13:30,741 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-292009172] 12:13:30,741 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:30,741 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1797050231] 12:13:30,741 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1797050231] 12:13:30,741 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:30,741 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,742 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,742 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,742 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:30,742 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:30,742 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,742 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,743 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:13:30,743 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,743 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,743 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,743 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,743 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:30,743 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:30,744 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:30,744 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:30,748 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:30,748 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:30,749 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:30,762 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:30,763 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,763 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,763 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:13:30,763 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,763 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,764 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,764 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,764 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,764 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:30,764 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:30,765 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:30,765 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,765 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,765 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:13:30,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,765 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,765 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,765 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:30,766 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:30,766 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:30,766 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:30,766 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:30,772 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:30,772 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:30,773 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:30.775] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:30,791 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:30,792 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,792 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:30,792 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:13:30,792 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,792 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 12:13:30,793 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 12:13:30,793 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1421cf9c 12:13:30,793 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1421cf9c 12:13:30,793 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 12:13:30,793 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 12:13:30,793 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,793 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 12:13:30,793 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56a6e58e 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56a6e58e 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:30,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:31,365 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 12:13:31,365 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 12:13:31,365 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 572.3 ms 12:13:31,365 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 12:13:31,365 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 12:13:31,365 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 12:13:31,365 PM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 572.2 ms 12:13:31,365 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 12:13:31,365 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 12:13:31,365 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 12:13:31,366 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:31,366 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 12:13:31,366 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:31,366 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:31,366 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:31,367 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:31,367 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 12:13:31,368 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:31,368 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:31,368 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 12:13:31,368 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 12:13:31,368 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7959d02a 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7959d02a 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d8f39f4 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d8f39f4 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 12:13:31,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:31,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:31,369 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:31,685 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 12:13:31,685 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 12:13:31,685 PM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 317.3 ms 12:13:31,685 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 317.4 ms 12:13:31,685 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 12:13:31,685 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 12:13:31,685 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 12:13:31,685 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 12:13:31,685 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 12:13:31,685 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 12:13:31,686 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:31,686 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 12:13:31,686 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:31,686 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 12:13:31,687 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:31,687 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:31,688 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:31,689 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:31,716 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:32,709 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:32.711] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:32,723 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:32,730 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5474836748640601037] 12:13:32,730 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:32,732 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:32,732 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:32,746 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:32,747 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:32,748 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,750 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:32,750 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:32,750 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,751 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:32,754 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,764 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2145640072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,765 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:32,766 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:32,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,767 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:32,767 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,767 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,769 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@41957d73 12:13:32,769 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 12:13:32,770 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:32,770 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,770 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,771 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,771 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,772 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-959188922 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,772 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-959188922] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,772 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-203152650] 12:13:32,773 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-203152650] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,773 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,773 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,773 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,773 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:32,773 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,773 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#136440600] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,773 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,774 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-520006658 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,774 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-520006658] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,774 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#124389215] 12:13:32,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#124389215] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,774 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,774 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,774 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,774 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#136440600] 12:13:32,777 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,778 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1554072554] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:32,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 12:13:32,781 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,781 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,781 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:32,781 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:32,781 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,781 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,781 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,784 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,803 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 12:13:32,803 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,803 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:32,804 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,804 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,804 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,804 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,804 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,804 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,805 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:32,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,805 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 12:13:32,806 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,806 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,806 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:32,806 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,806 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,806 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,806 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@65992073 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:32,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,808 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,808 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,808 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1588755605] 12:13:32,808 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,808 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,808 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,809 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#274337751] 12:13:32,809 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,810 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1588755605] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,810 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#1429661569] 12:13:32,810 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#274337751] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,810 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,810 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,811 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#181176921] 12:13:32,811 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-913382683 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,811 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:32,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:32,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,812 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,812 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1497345792 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,812 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,812 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,812 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,821 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:32,830 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4274187963428494025] 12:13:32,831 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:32,833 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:32,833 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:32,835 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:32,835 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:32,845 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,850 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,853 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,861 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1439259166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,864 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:32,872 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:32,872 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,872 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,875 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,876 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@22e0a3c0 12:13:32,876 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 12:13:32,876 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:32,876 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,876 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,878 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,880 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,883 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,883 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,883 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1433521164] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,883 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1275424911] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,884 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,884 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1080121482 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,884 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1340555293 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,884 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1080121482] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,884 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1340555293] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,885 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#2043905898] 12:13:32,885 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#2043905898] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,885 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1747431824] 12:13:32,885 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1747431824] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,885 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,885 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2116761178] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,885 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,886 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,886 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,886 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,886 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:32,886 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 12:13:32,887 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,887 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,887 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:32,887 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,887 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,887 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,888 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,890 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:32,891 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,891 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,891 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,892 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 12:13:32,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 12:13:32,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:32,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,893 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,893 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:32,893 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,893 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,894 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,894 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,894 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,894 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:13:32,894 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,894 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,894 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,894 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b7dd7bc 12:13:32,894 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:32,894 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,895 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,895 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,897 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1205878168] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,898 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,898 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#913209543] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,899 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:32,899 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,900 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1346899261 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,900 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1689410380] 12:13:32,900 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1123742096 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,900 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#253340429] 12:13:32,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:32,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,902 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,902 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:32,902 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,902 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,902 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,903 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,903 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,903 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,904 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,905 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,911 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,911 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,911 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,911 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,911 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 12:13:32,922 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7525294477596956213] 12:13:32,923 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:32,925 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:32,925 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:32,926 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:32,926 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:32,933 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,935 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,950 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:32,950 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,953 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:32,958 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:32,958 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-136405868] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,963 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:32,963 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,964 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:32,964 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,964 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,967 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,967 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6e5196 12:13:32,968 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 12:13:32,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:32,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,968 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#801431420] 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1372005302] 12:13:32,969 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:32,969 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,970 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,970 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,970 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#801431420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1372005302] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#419433311] 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2047934637 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#2014112090] 12:13:32,971 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1564854191] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,971 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1564854191 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,972 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:32,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:32,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 12:13:32,977 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,977 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,977 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#419433311] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,977 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2047934637] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,977 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#2014112090] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,977 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-945834988] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,978 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,979 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 12:13:32,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:32,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,979 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:32,980 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,980 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:32,980 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,981 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,981 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@456bcdc7 12:13:32,981 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:32,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,982 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,982 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,982 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1303456571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,982 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-178180146 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,983 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1557769746] 12:13:32,983 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-178180146] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:32,983 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,984 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:13:32,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,984 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,984 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,984 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,984 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,984 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,985 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1303456571] 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1074483394] 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:32,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,985 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,985 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,985 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:32,986 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 12:13:32,986 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1984614866 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,986 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1295943026] 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 12:13:32,987 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:13:32,987 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:32,987 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:32,987 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 12:13:32,988 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,988 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:32,988 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 12:13:33,150 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 12:13:33,150 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:33,150 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:33,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:33,151 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:33,151 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 12:13:33,151 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 12:13:33,151 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 12:13:33,151 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 12:13:33,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 12:13:33,222 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 12:13:33,222 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 12:13:33,222 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 12:13:33,501 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,501 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,501 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 12:13:33,541 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,541 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,541 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 12:13:33,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,561 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 12:13:33,611 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,611 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,611 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 12:13:33,752 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:33,752 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:33,752 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:33,752 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:33,752 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,753 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:33,761 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,761 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,762 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,762 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,867 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,868 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,891 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,891 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,891 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 12:13:33,891 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,891 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,893 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,894 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:13:33,894 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:13:33,894 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:33,894 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 12:13:33,895 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,895 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 12:13:33,895 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,895 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 12:13:33,896 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:13:33,896 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,896 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,896 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,897 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,897 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,897 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,897 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,897 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,898 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,899 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,899 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,900 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,900 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,901 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,901 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,901 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,901 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,901 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,901 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,901 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 12:13:33,902 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,902 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,902 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 4 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,903 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,904 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,904 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:13:33,904 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,905 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,905 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 12:13:33,905 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,905 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,906 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,906 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,906 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 12:13:33,906 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,906 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,906 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,907 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,908 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,908 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,908 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,909 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,909 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,909 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,909 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,909 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,910 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 12:13:33,910 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,910 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,910 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,911 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,912 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,920 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,920 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,920 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 12:13:33,920 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,920 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,921 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,921 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,921 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,921 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,921 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,922 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,922 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,921 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,922 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,922 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,922 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,922 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,922 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,922 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,922 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,922 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 12:13:33,923 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,923 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,924 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 12:13:33,924 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:13:33,924 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 12:13:33,924 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,924 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,924 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,925 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,925 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,925 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,925 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,925 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,925 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,925 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,926 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,926 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,926 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,926 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,926 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,926 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,926 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,926 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,927 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,928 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 12:13:33,928 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 12:13:33,928 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,928 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,929 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 4 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,930 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,930 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,945 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:13:33,971 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 12:13:33,971 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,971 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 12:13:33,971 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,971 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,973 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,973 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:33,973 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 12:13:33,973 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 12:13:33,974 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 12:13:33,974 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,974 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 12:13:33,974 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,974 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,974 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,975 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 12:13:33,975 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,975 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 12:13:33,975 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,975 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,975 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 12:13:33,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,976 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,976 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,976 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,976 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 12:13:33,976 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,976 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 12:13:33,976 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,977 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 12:13:33,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,977 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 12:13:33,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,978 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,978 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,979 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]] - lastApplied: 1, lastIndex: 1 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 12:13:33,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,980 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 12:13:33,980 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 12:13:33,980 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 12:13:33,980 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 12:13:33,980 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:33,980 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 12:13:33,980 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2145640072] 12:13:33,980 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:33,980 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:33,980 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,981 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:33,981 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,981 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 12:13:33,981 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,981 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,981 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,981 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,981 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 12:13:33,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] 12:13:33,982 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 12:13:33,982 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5]]] 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5] is present: existingEntryTerm: 4 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 12:13:33,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 12:13:33,982 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 12:13:33,982 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@58b0d3c5 12:13:33,982 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 12:13:33,982 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:13:33,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 12:13:33,982 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:33,982 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:13:33,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:33,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:13:33,983 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,983 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,983 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 12:13:33,983 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:33,983 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 12:13:33,983 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 12:13:33,983 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=18a37f3c-fb2f-480f-9f51-3d49f8007f30}, logIndex: 2, isSendImmediate: true 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 12:13:33,984 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 12:13:33,984 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 12:13:33,984 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,984 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 12:13:33,984 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=67cfc2af-f90c-402e-8e7f-639e269b7c42}, logIndex: 2, isSendImmediate: true 12:13:33,984 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,984 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,985 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,985 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 12:13:33,985 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,985 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 12:13:33,985 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 12:13:33,985 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 12:13:33,985 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,983 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=0a174809-8660-4236-af05-110d8ea14ddc}, logIndex: 2, isSendImmediate: true 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,985 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,985 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 12:13:33,985 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,985 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,986 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,986 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,986 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 12:13:33,986 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,986 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,986 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,986 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,986 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,986 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,987 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,987 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,988 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,988 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,988 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,988 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,988 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,989 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]], state: Idle 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=6be73095-ba90-4020-b90b-10add747cbba}, logIndex: 2, isSendImmediate: true 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,989 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 12:13:33,990 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:33,990 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 12:13:33,990 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 12:13:33,990 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 12:13:33,990 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2145640072] 12:13:33,990 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,990 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:33,990 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,990 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,990 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:33,990 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,990 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,990 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,990 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:33,990 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,990 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,991 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:33,991 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,991 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,991 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,991 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2145640072] 12:13:33,991 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 12:13:33,991 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#136440600] 12:13:33,991 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#136440600] 12:13:33,991 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1554072554] 12:13:33,991 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 12:13:33,991 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1588755605] 12:13:33,992 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1588755605] 12:13:33,992 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1554072554] 12:13:33,992 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 12:13:33,992 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#274337751] 12:13:33,992 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#274337751] 12:13:33,992 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1439259166] 12:13:33,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:33,992 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1275424911] 12:13:33,992 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1275424911] 12:13:33,992 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,993 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:33,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 12:13:33,994 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 12:13:33,994 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:34,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1275424911] 12:13:34,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1439259166] 12:13:34,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 12:13:34,43 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1433521164] 12:13:34,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1433521164] 12:13:34,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2116761178] 12:13:34,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 12:13:34,43 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1205878168] 12:13:34,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1205878168] 12:13:34,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2116761178] 12:13:34,44 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 12:13:34,44 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#913209543] 12:13:34,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#913209543] 12:13:34,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-136405868] 12:13:34,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:34,44 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#801431420] 12:13:34,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#801431420] 12:13:34,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-136405868] 12:13:34,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 12:13:34,45 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1372005302] 12:13:34,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1372005302] 12:13:34,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-945834988] 12:13:34,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 12:13:34,45 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1303456571] 12:13:34,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1303456571] 12:13:34,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-945834988] 12:13:34,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 12:13:34,45 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1074483394] 12:13:34,45 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1074483394] 12:13:34,46 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-2145640072] 12:13:34,46 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:34,47 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:34,47 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1287806056] 12:13:34,47 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:34,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,48 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 12:13:34,48 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,48 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,48 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,48 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,49 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,48 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,49 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:34,49 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:34,49 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:34,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,50 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:13:34,50 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,50 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,50 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,50 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,50 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,50 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,50 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:34,50 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:34,52 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:34,52 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:34,56 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:34,56 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:34,56 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:34,56 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:34,59 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:34,71 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:34,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,71 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:13:34,71 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,72 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,72 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,72 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,72 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:34,72 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:34,72 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:34,72 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,72 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,73 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:13:34,73 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,73 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:34,73 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,73 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:34,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:34,73 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:34,73 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:34,74 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:34,74 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:34,601 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 12:13:34,601 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 12:13:34,601 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 12:13:34,601 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 12:13:34,772 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:35,71 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [7525294477596956213] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:35,91 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4274187963428494025] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:35,94 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:35,94 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:35,96 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:35.098] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:35,102 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@525d515f 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@525d515f 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,103 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@597cee42 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@597cee42 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,103 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,103 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 12:13:35,521 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 12:13:35,521 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 418.3 ms 12:13:35,521 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 12:13:35,521 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 12:13:35,521 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 12:13:35,522 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:35,522 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:35,571 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 12:13:35,571 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 467.5 ms 12:13:35,571 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 12:13:35,571 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 12:13:35,571 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 12:13:35,572 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 12:13:35,572 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:35,572 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:35,573 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:35,573 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 12:13:35,573 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:35,573 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 12:13:35,574 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3d64fd29 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3d64fd29 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,574 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 12:13:35,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2599c412 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2599c412 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:35,574 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 566.9 ms 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 567.6 ms 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 12:13:36,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 12:13:36,142 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:36,142 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 12:13:36,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:36,142 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 12:13:36,142 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:36,142 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:36,143 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:36,143 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:36,146 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:36.147] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:36,167 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:36,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2043780707517445359] 12:13:36,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:36,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:36,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:36,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:36,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:36,191 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,198 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:36,198 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:36,198 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:36,208 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,211 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 12:13:36,214 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:36,216 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1090700783] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,218 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:36,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,219 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,219 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:36,219 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,219 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,224 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:36,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7fca9caf 12:13:36,225 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 12:13:36,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:36,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,225 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619] 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-643273632] 12:13:36,226 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 12:13:36,227 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,227 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,227 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardLeaderReplica, persistent : true 12:13:36,227 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,228 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,228 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-643273632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,228 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,229 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:36,230 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1750618954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,230 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#430247223 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,230 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#2115261713] 12:13:36,232 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,232 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:36,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 12:13:36,232 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#706122902] 12:13:36,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,232 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,232 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:36,232 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,232 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,233 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:36,233 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,233 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,233 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:36,233 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 12:13:36,233 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,233 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,233 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,233 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 12:13:36,233 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 12:13:36,235 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,235 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1750618954 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#1330412241] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#430247223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#2115261713] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,235 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#706122902] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,235 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 12:13:36,236 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 12:13:36,237 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@156e8b62 12:13:36,237 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 12:13:36,237 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:36,237 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 12:13:36,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#1094019625] 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,238 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#872862529] 12:13:36,239 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 12:13:36,239 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,239 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-791201654] 12:13:36,239 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-658297266 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,239 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1045126745] 12:13:36,239 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#580069785 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,240 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#1094019625] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,240 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#872862529] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,244 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,244 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,244 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,244 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,244 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,244 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:36,244 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 12:13:36,244 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,244 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,244 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,244 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,244 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,244 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,244 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 12:13:36,245 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:36,245 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 12:13:36,245 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,245 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 12:13:36,247 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:36,256 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1680656036658088664] 12:13:36,256 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:36,262 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:36,262 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:36,262 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:36,262 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:36,269 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,270 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,272 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 12:13:36,284 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#992500604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,285 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:36,293 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:36,293 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,293 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:36,296 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:36,301 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3e200136 12:13:36,301 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 12:13:36,302 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:36,302 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,302 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,304 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,306 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,309 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:36,310 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-676167753] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,311 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 12:13:36,311 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1818408210 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,311 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1809398454] 12:13:36,311 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,312 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardLeaderReplica, persistent : true 12:13:36,312 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#627421603 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:36,312 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#1809348710] 12:13:36,312 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:36,315 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,315 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1818408210] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,315 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1809398454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,315 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-1119693959] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,315 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#627421603] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,315 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#1809348710] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,315 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:36,315 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,316 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,316 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 12:13:36,317 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@34d86a46 12:13:36,318 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 12:13:36,318 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:36,318 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,318 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,318 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 12:13:36,319 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,320 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 12:13:36,320 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#2126632741 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#982525745] 12:13:36,321 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1107986369 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,321 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1360988008] 12:13:36,321 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#1005001766] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,321 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#624619020] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,321 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,322 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,323 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,323 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,323 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:36,323 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 12:13:36,324 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,324 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,324 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,324 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 12:13:36,324 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 12:13:36,331 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,331 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,331 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,331 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,331 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,331 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:13:36,331 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 12:13:36,331 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:13:36,331 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 12:13:36,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,332 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,332 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,332 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,332 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 12:13:36,332 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 12:13:36,332 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 12:13:36,332 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,332 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,333 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:36,333 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 12:13:36,333 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,333 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,333 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,333 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,333 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 12:13:36,335 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 12:13:36,342 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:36,344 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,347 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,348 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:36,348 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,348 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,348 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,348 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,348 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:36,348 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,353 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-9170400650295974191] 12:13:36,353 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:36,357 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:36,357 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:36,358 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:36,358 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:36,368 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,369 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,371 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 12:13:36,374 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1532129469] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,376 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:36,384 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:36,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:36,387 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:36,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@211ae0c5 12:13:36,388 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 12:13:36,388 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,390 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:36,394 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:36,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,395 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:36,395 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 12:13:36,395 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1398177062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,396 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,396 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-643768018] 12:13:36,396 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardLeaderReplica, persistent : true 12:13:36,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1261577972 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,396 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#-1672614317 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:36,396 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1040388589] 12:13:36,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#155941402] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,397 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:36,412 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:36,413 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,413 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1261577972] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-643768018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#-1672614317] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1040388589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-711348989] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,417 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 12:13:36,417 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,417 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,417 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,418 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,419 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:36,419 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 12:13:36,420 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:36,420 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:36,420 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:36,420 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 12:13:36,422 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:36,422 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,422 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,422 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,422 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:36,422 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,422 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,422 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,423 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,423 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,423 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4aa991d6 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,424 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,425 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 12:13:36,425 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 12:13:36,425 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#-523075939] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,425 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-672652393] 12:13:36,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-421593313] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:36,426 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-421593313 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,426 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,427 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 12:13:36,427 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-297574377] 12:13:36,427 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1218690458 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,428 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:36,428 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,428 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,428 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 12:13:36,428 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:13:36,428 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 12:13:36,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,428 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 12:13:36,428 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 12:13:36,428 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,428 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:13:36,428 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 12:13:36,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 12:13:36,428 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 12:13:36,428 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 12:13:36,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 12:13:36,429 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,619 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 12:13:36,619 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 12:13:36,627 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:36,627 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 12:13:36,627 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,628 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 12:13:36,628 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:36,628 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 12:13:36,628 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:36,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 12:13:36,628 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 12:13:36,628 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 12:13:36,958 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:36,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:36,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:36,958 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:36,958 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 12:13:36,958 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 12:13:37,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:37,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,18 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 12:13:37,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:37,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,38 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 12:13:37,199 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:37,199 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,200 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,207 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:37,297 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,297 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,298 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:37,298 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,298 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 12:13:37,298 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,299 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,301 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:37,301 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:37,301 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 12:13:37,301 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,302 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,303 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:37,303 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,303 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:37,303 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 12:13:37,303 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 12:13:37,303 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,303 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,303 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 12:13:37,303 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 12:13:37,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,303 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,303 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 12:13:37,304 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,304 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,304 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,304 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,357 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:37,357 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,357 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 12:13:37,358 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,358 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,359 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:37,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,361 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,361 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:37,361 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:37,361 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 12:13:37,361 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 12:13:37,361 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,361 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 12:13:37,362 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,362 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:37,362 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,362 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 12:13:37,362 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 12:13:37,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,363 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,363 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,363 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,363 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,372 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:13:37,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:37,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,408 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 12:13:37,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,409 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,410 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,411 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:37,411 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:37,411 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 12:13:37,412 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 12:13:37,412 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 12:13:37,413 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,413 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,413 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 12:13:37,413 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 12:13:37,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,413 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 12:13:37,413 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,414 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 12:13:37,414 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,414 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,414 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,414 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 12:13:37,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,418 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 12:13:37,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,419 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:37,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:37,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:37,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 12:13:37,421 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 12:13:37,421 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 12:13:37,421 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1090700783] 12:13:37,421 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:37,421 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619] 12:13:37,421 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,422 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,422 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 12:13:37,422 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 12:13:37,422 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,422 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 12:13:37,422 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 12:13:37,422 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619] 12:13:37,422 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,422 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#992500604] 12:13:37,423 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,423 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 12:13:37,423 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:37,423 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:37,423 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,423 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,423 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1532129469] 12:13:37,423 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:37,423 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1398177062] 12:13:37,423 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1398177062] 12:13:37,423 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-1, datastoreType Config 12:13:37,423 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@75422e4b] 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onRemoveServer: RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'}, state: Idle 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]] 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=af76e871-0c1b-4e3d-935f-a86d7dfecc22}, logIndex: 0, isSendImmediate: true 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,424 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 12:13:37,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 12:13:37,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: Returning OK for operation RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'} 12:13:37,425 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 12:13:37,425 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 12:13:37,425 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 12:13:37,425 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#992500604] 12:13:37,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,425 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:37,425 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 12:13:37,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 12:13:37,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:37,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 12:13:37,426 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,426 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testRemoveShardLeaderReplica, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 12:13:37,426 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 12:13:37,426 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,426 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testRemoveShardLeaderReplica, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:37,426 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 12:13:37,426 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 12:13:37,426 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Setting last applied to 0 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,427 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:37,428 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 12:13:37,428 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 12:13:37,428 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:37,428 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 12:13:37,428 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 12:13:37,428 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 12:13:37,428 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 12:13:37,428 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619] with 10000 ms timeout 12:13:37,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 12:13:37,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 12:13:37,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 12:13:37,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:37,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 12:13:37,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 12:13:37,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 12:13:37,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 12:13:37,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4d55a278 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4d55a278 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 12:13:37,429 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 12:13:37,430 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 12:13:37,430 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,430 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Successfully deleted prior snapshots 12:13:37,430 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 12:13:37,430 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 12:13:37,431 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 12:13:37,431 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 3 - updating term 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=null 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 3 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 12:13:37,434 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 12:13:37,434 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 12:13:37,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-2-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,435 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 12:13:37,435 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 12:13:37,435 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 12:13:37,435 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 12:13:37,435 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,436 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 12:13:37,479 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,479 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#992500604] 12:13:37,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:37,479 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,479 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#433608231] 12:13:37,481 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1532129469] 12:13:37,481 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:37,481 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1398177062] 12:13:37,481 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1398177062] 12:13:37,481 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1090700783] 12:13:37,481 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:37,481 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 12:13:37,481 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:37,482 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,482 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,482 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 12:13:37,482 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onShutDown 12:13:37,482 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 12:13:37,482 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardLeaderReplica 12:13:37,482 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,482 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:37,482 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:37,483 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:37,483 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:37,483 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 12:13:37,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,483 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:13:37,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 12:13:37,483 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 12:13:37,483 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 12:13:37,484 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:37,484 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 12:13:37,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:37,484 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:37,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:37,485 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:37,485 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:37,489 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:37,490 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:37,490 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:37,490 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:37,492 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:37,512 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:37,512 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,512 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,512 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:13:37,512 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 12:13:37,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 12:13:37,513 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onShutDown 12:13:37,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11f51df6 12:13:37,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11f51df6 12:13:37,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 12:13:37,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 12:13:37,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:37,513 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 12:13:37,513 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardLeaderReplica 12:13:37,513 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:37,620 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 12:13:37,621 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 12:13:37,729 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 12:13:37,730 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 12:13:37,739 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 4 12:13:37,740 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 4 in "AppendEntriesReply [term=4, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5]" message is greater than leader's term 3 - switching to Follower 12:13:37,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=null 12:13:37,740 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 4 12:13:37,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 12:13:37,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 12:13:37,741 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 12:13:37,741 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 12:13:37,741 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:37,741 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 12:13:37,941 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:38,50 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:38,219 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:38,260 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:38,370 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:38,498 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-9170400650295974191] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:38,501 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-9170400650295974191] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:38,580 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:38,689 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:38,901 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:39,11 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:13:39,221 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:39,330 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:39,448 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 12:13:39,448 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.020 s 12:13:39,449 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 12:13:39,449 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 12:13:39,449 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:39,449 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-79773619] 12:13:39,541 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:39,649 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:39,860 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:39,970 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:40,181 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:40,290 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:40,500 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:40,611 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:40,820 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:40,930 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:41,141 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:13:41,250 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:41,460 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:41,570 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:41,781 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:41,890 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:42,103 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 12:13:42,209 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:42,418 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 12:13:42,418 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 12:13:42,420 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:42,421 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 12:13:42,421 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 12:13:42,421 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:42,479 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 12:13:42,479 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:42,528 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 12:13:42,528 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 12:13:42,531 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:42,531 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 12:13:42,531 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 12:13:42,531 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 12:13:42,741 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:42,850 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 12:13:43,61 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:43,170 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:43,319 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 12:13:43,319 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:43,321 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:43,321 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:43,323 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:43,323 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:43,380 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:43,401 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 12:13:43,495 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 324, lastApplied : -1, commitIndex : -1 12:13:43,701 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:13:43,810 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 12:13:44,20 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:44,129 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:44,340 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:44,450 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:44,660 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:44,769 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:44,980 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:44,983 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:13:45,90 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:45,116 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 12:13:45,301 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 12:13:45,410 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:45,621 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:45,730 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:45,940 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:46,50 PM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:46,261 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:46,369 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 12:13:46,580 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 12:13:46,690 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 12:13:46,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 12:13:46,701 PM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.188 s 12:13:46,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 12:13:46,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer failed - sending PoisonPill 12:13:46,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer expired 12:13:46,701 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 12:13:46,702 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:46,702 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:46,702 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:46,702 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 12:13:46,703 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:46,703 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:46,703 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:13:46,703 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 12:13:46,703 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 12:13:46,703 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 12:13:46,703 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 12:13:46,703 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:46,703 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:46,703 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:46,704 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:46,704 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:46,704 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:47,538 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:47,539 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 12:13:47,539 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:47,539 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:47,539 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:13:47,539 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:47,539 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 12:13:47,539 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:47,648 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:47,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 12:13:47,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:47,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:47,649 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 12:13:47,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 12:13:47,649 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 12:13:47,649 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 12:13:47,718 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1680656036658088664] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:47,723 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:47,723 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:47,724 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:47.727] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:47,732 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 12:13:47,732 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:47,732 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 12:13:47,732 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onShutDown 12:13:47,732 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardLeaderReplica 12:13:47,732 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:47,732 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:47,733 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:47,733 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:47,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 12:13:47,734 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 12:13:47,734 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 12:13:47,734 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:47,734 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:47,735 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:47,735 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:47,736 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:47.738] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:47,757 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:47,764 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3998094221760912905] 12:13:47,764 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:47,767 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:47,767 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:47,767 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:47,767 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:47,782 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:47,782 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:47,783 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:47,783 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,785 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,788 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,792 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:47,793 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-231507790] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,800 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:47,800 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,800 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,801 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:47,801 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,801 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,802 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 12:13:47,803 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1f362dac 12:13:47,803 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 12:13:47,803 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:47,803 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,803 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,804 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 12:13:47,804 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 12:13:47,805 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-2083100001] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,805 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1129131387 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,805 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1129131387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#1678638027] 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-2083100001] 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,805 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1431411831] 12:13:47,805 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 12:13:47,806 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1794773608 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,806 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#1678638027] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,806 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:47,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 12:13:47,807 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 12:13:47,807 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 12:13:47,808 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1431411831] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,808 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1794773608] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,808 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 12:13:47,808 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 12:13:47,808 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:47,808 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:47,808 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 12:13:47,808 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 12:13:47,808 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,809 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,810 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#1206796880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:47,811 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1018477611] 12:13:47,811 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 12:13:47,811 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1018477611] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,811 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 12:13:47,811 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,812 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:47,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 12:13:47,815 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 12:13:47,815 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7eb9d6f9 12:13:47,815 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 12:13:47,816 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 12:13:47,816 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:47,816 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,817 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 12:13:47,817 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#1466492544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,817 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 12:13:47,817 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#1235282941] 12:13:47,818 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,818 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#1235282941] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#1466492544] 12:13:47,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,818 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#1398061808 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,818 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,818 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 12:13:47,819 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#1253271672 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,819 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1569733026] 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:47,819 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 12:13:47,819 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#349742453] 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,820 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,820 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 12:13:47,820 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 12:13:47,820 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 12:13:47,820 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 12:13:47,821 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 12:13:47,821 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 12:13:47,821 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,821 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 12:13:47,821 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 12:13:47,824 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 12:13:47,824 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 12:13:47,824 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 12:13:47,824 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:47,824 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 12:13:47,824 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 12:13:47,824 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,825 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 12:13:47,825 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 12:13:47,825 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 12:13:47,825 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 12:13:47,827 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-1=false} 12:13:47,828 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 12:13:47,828 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:47,828 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]], state: Idle 12:13:47,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:47,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Returning INVALID_REQUEST for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] 12:13:47,829 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,829 PM [cluster-test-akka.actor.default-dispatcher-21] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard cars java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$11.onComplete(ShardManager.java:1558) at akka.dispatch.OnComplete.internal(Future.scala:300) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 12:13:47,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-231507790] 12:13:47,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:47,830 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-2083100001] 12:13:47,830 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-2083100001] 12:13:47,830 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 12:13:47,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:13:47,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 12:13:47,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4dd01eb2 12:13:47,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4dd01eb2 12:13:47,830 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 12:13:47,831 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 12:13:47,831 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6dd34e49 12:13:47,831 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6dd34e49 12:13:47,831 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,831 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 12:13:47,831 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:47,831 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:47,831 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 12:13:47,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,832 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:47,832 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 12:13:47,832 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6314bd83 12:13:47,832 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6314bd83 12:13:47,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 12:13:47,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1c9abe21 12:13:47,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1c9abe21 12:13:47,832 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,832 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 12:13:47,833 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 12:13:47,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 12:13:47,833 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:47,833 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:47,834 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:47,834 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:47,835 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:47,853 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:47,867 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3482302793761866319] 12:13:47,867 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 12:13:47,869 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 12:13:47,869 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 12:13:47,870 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:47,870 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:47,881 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,881 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 12:13:47,882 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 12:13:47,882 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 12:13:47,882 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,884 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 12:13:47,893 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1373110935] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,893 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 12:13:47,896 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 12:13:47,896 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,896 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,896 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:47,896 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,896 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,898 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:47,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@24d5b619 12:13:47,899 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 12:13:47,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:47,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,899 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,900 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,900 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,900 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1701872129] 12:13:47,900 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,901 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,901 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,901 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-1031240379] 12:13:47,901 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1701872129] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,901 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,901 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,901 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,901 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-1031240379] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,901 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1446312983 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,901 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1446312983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,902 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#1760733366] 12:13:47,902 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#1760733366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,903 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:47,904 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1238559066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,904 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 12:13:47,906 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,907 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 12:13:47,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,907 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,907 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:47,907 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,907 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,907 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,907 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,908 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 12:13:47,908 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 12:13:47,908 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 12:13:47,908 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 12:13:47,908 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@18aacc3d 12:13:47,909 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:47,909 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,915 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,915 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,915 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,915 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 12:13:47,915 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,915 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,915 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#645550033] 12:13:47,915 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,916 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,916 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-545754913] 12:13:47,916 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,916 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,917 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1885653393 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,917 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1598240106] 12:13:47,917 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1648585552] 12:13:47,918 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#645550033] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-545754913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1885653393] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1598240106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 12:13:47,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,921 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,921 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 12:13:47,921 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:47,921 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,921 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#312182324 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,922 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,922 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 12:13:47,922 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,922 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1253135705 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,922 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#2021690713] 12:13:47,922 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,923 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,923 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 12:13:47,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 12:13:47,923 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,923 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,923 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,923 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:47,923 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,924 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 12:13:47,932 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7566518294427097289] 12:13:47,933 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 12:13:47,935 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:47,935 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 12:13:47,935 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:47,935 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:47,948 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,949 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,952 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 12:13:47,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-990280469] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,959 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 12:13:47,962 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 12:13:47,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,964 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:47,965 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@210aa9ca 12:13:47,965 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 12:13:47,966 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,967 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:47,967 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,967 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:47,969 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:47,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,971 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,971 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#322390394] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,972 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#618047692] 12:13:47,972 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1484566769 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:47,972 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#618047692] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,972 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1484566769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,972 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#-1397977431] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,972 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,972 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1871574968 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:47,973 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1871574968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,973 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-1623440457] 12:13:47,973 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-1623440457] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,973 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,973 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1156408786] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,974 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 12:13:47,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,974 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,975 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,975 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 12:13:47,975 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 12:13:47,984 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 12:13:47,984 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,985 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,985 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,985 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 12:13:47,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 12:13:47,985 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,985 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,985 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,986 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 12:13:47,986 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,986 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,986 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 12:13:47,986 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,987 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 12:13:47,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@215e215c 12:13:47,988 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 12:13:47,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:47,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,990 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,990 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,990 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 12:13:47,991 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:47,992 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1710849361 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:47,992 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#185307960] 12:13:47,992 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#-1253717237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,992 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#-307329594] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:47,992 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#1343819919] 12:13:47,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,994 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 12:13:47,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 12:13:47,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:47,994 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:47,994 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 12:13:47,994 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 12:13:47,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:47,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:47,994 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,994 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:47,994 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 12:13:47,995 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#590486735 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:47,995 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:47,995 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:47,995 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 12:13:48,0 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 12:13:48,5 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 12:13:48,8 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:48,10 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:48,10 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,10 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:48,10 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,11 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,19 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6192857391692575055] 12:13:48,19 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 12:13:48,20 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:48,20 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 12:13:48,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 12:13:48,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 12:13:48,32 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:48,33 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:48,35 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 12:13:48,44 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 12:13:48,44 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1506526463] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,48 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 12:13:48,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,51 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:48,52 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@69039562 12:13:48,52 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 12:13:48,53 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:48,54 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 12:13:48,56 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:48,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:48,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,57 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 12:13:48,57 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-2140348688] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,57 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 12:13:48,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,58 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 12:13:48,58 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1548227000] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,58 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-300286915] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,58 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1297333618 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,58 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1885544810 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,59 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 12:13:48,59 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,59 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1885544810] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,59 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1297333618] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,59 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,62 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:48,63 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 12:13:48,65 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:48,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@63fbfe12 12:13:48,66 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 12:13:48,66 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 12:13:48,66 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 12:13:48,66 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:48,66 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#767668175] 12:13:48,66 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#767668175] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,67 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:48,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 12:13:48,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,67 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 12:13:48,68 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 12:13:48,68 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:48,68 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#504821752 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,68 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#1835452361] 12:13:48,68 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#273555222] 12:13:48,68 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1773849465 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,69 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 12:13:48,69 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-204553] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,69 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#-2066922069] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,69 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1773849465] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 12:13:48,72 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1399681880] 12:13:48,73 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 12:13:48,73 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:48,73 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:48,73 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:48,73 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:48,73 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,73 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,73 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 12:13:48,73 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:48,73 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 12:13:48,73 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 12:13:48,73 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 12:13:48,73 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 12:13:48,73 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 12:13:48,74 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 12:13:48,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 12:13:48,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 12:13:48,76 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,76 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,76 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 12:13:48,76 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:48,76 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 12:13:48,76 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 12:13:48,76 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 12:13:48,76 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,76 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,280 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:48,280 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:48,280 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 12:13:48,280 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 12:13:48,280 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 12:13:48,280 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 12:13:48,280 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,280 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,280 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 12:13:48,280 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 12:13:48,280 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:48,281 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:48,281 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,281 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,281 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,281 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,281 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 12:13:48,281 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 12:13:48,281 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 12:13:48,281 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 12:13:48,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,321 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 12:13:48,321 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 12:13:48,340 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 12:13:48,340 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 12:13:48,661 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:48,661 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,661 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 12:13:48,670 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:48,670 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,670 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 12:13:48,690 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:48,690 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,690 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 12:13:48,721 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:48,721 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,721 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 12:13:48,881 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:48,881 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,881 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,881 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 12:13:48,882 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:48,882 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,882 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,883 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,883 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,883 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:48,890 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,890 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 12:13:48,966 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,966 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 12:13:48,967 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,967 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:48,967 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:48,967 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,0 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:49,0 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,0 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 12:13:49,0 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,0 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,2 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:49,2 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 12:13:49,2 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:49,2 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:49,2 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:49,3 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:49,3 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:49,3 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,3 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 12:13:49,4 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,4 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 12:13:49,5 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,5 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,5 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 12:13:49,6 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,6 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 12:13:49,6 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 12:13:49,6 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,6 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,6 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 12:13:49,6 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,43 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 12:13:49,70 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:49,70 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,70 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 12:13:49,70 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,70 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:49,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 12:13:49,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:49,71 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:49,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 12:13:49,72 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:49,72 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:49,72 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,72 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:49,72 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 12:13:49,72 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:49,72 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:49,72 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 12:13:49,73 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:49,73 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:49,73 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:49,73 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 12:13:49,73 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 12:13:49,74 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,74 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 12:13:49,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 12:13:49,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,75 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 12:13:49,75 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,75 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,76 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,76 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 12:13:49,76 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,76 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 12:13:49,90 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:49,90 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,90 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 12:13:49,90 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,90 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,92 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,92 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 12:13:49,93 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 12:13:49,94 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 12:13:49,94 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 12:13:49,94 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,94 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 12:13:49,94 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 12:13:49,94 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 12:13:49,94 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:49,95 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,95 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,95 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 12:13:49,95 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 12:13:49,95 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 12:13:49,95 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1373110935] 12:13:49,96 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:49,96 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1701872129] 12:13:49,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1701872129] 12:13:49,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,96 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 12:13:49,96 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 12:13:49,96 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-990280469] 12:13:49,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:49,96 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 12:13:49,96 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#322390394] 12:13:49,96 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 12:13:49,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#322390394] 12:13:49,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1506526463] 12:13:49,97 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:49,97 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1548227000] 12:13:49,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1548227000] 12:13:49,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 12:13:49,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:49,97 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-2=false, member-3=false} 12:13:49,97 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 12:13:49,98 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:49,98 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,98 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 12:13:49,98 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,98 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard)] 12:13:49,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 12:13:49,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 12:13:49,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 12:13:49,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]] 12:13:49,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9255bdc9-614b-4912-98f3-422c5331c434}, logIndex: 0, isSendImmediate: true 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Setting last applied to 0 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] has been successfully replicated to a majority of followers 12:13:49,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 12:13:49,101 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 12:13:49,101 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING]] 12:13:49,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 12:13:49,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 12:13:49,102 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 12:13:49,102 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,102 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 12:13:49,102 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard cars 12:13:49,103 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,103 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 12:13:49,103 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 12:13:49,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 12:13:49,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:49,104 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 12:13:49,104 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 12:13:49,104 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 12:13:49,104 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1373110935] 12:13:49,104 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 12:13:49,104 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1701872129] 12:13:49,104 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1701872129] 12:13:49,105 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-990280469] 12:13:49,105 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 12:13:49,105 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#322390394] 12:13:49,105 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#322390394] 12:13:49,105 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1506526463] 12:13:49,105 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 12:13:49,105 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1548227000] 12:13:49,105 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1548227000] 12:13:49,105 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:49,105 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,105 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,105 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 12:13:49,105 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,106 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,106 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,106 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,106 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,106 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,106 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 12:13:49,106 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 12:13:49,106 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 12:13:49,106 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,106 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,106 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 12:13:49,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,107 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,107 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,107 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,107 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,107 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 12:13:49,107 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 12:13:49,108 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:49,108 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:49,111 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:49,111 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:49,112 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:49,112 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:49,113 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 12:13:49,124 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:49,124 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,124 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,124 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 12:13:49,124 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,124 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,124 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 12:13:49,124 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:49,124 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,124 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,125 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 12:13:49,125 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 12:13:49,125 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 12:13:49,126 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,126 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,126 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 12:13:49,126 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,126 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 12:13:49,126 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:49,126 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,126 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,126 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:49,126 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 12:13:49,126 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 12:13:49,128 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:49,128 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:49,130 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:49,130 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,130 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 12:13:49,130 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,130 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,491 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:49,491 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,491 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 4 12:13:49,491 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,491 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,840 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:49,840 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,841 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 5 12:13:49,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:49,901 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 12:13:50,120 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-6192857391692575055] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:50,140 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7566518294427097289] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 12:13:50,147 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 12:13:50,150 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:50.152] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 12:13:50,174 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 12:13:50,175 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:50,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:50,176 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 12:13:50,176 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 12:13:50,176 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 12:13:50,176 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 12:13:50,176 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 12:13:50,176 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 12:13:50,176 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 12:13:50,177 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c1a795a 12:13:50,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@100b7f6e 12:13:50,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@100b7f6e 12:13:50,177 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c1a795a 12:13:50,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 12:13:50,177 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 12:13:50,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 12:13:50,177 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 12:13:50,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:50,177 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:50,178 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 12:13:50,178 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:50,230 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:50,231 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=6, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,231 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 6 12:13:50,231 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,232 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 12:13:50,600 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 424.0 ms 12:13:50,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 12:13:50,601 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 12:13:50,601 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 12:13:50,602 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 12:13:50,602 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:50,630 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 12:13:50,630 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=7, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,631 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 7 12:13:50,631 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=7, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,631 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=7, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,690 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 12:13:50,690 PM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 514.1 ms 12:13:50,690 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 12:13:50,690 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 12:13:50,690 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 12:13:50,691 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 12:13:50,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:50,691 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 12:13:50,691 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 12:13:50,692 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 12:13:50,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:50,693 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 12:13:50,693 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 12:13:50,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 12:13:50,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 12:13:50,693 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 12:13:50,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e8e3fc1 12:13:50,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e8e3fc1 12:13:50,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 12:13:50,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 12:13:50,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:50,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 12:13:50,694 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 12:13:50,694 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:51,310 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 12:13:51,310 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-oper-member-1: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-545754913]] after [600 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 12:13:51,311 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 12:13:51,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 12:13:51,311 PM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 617.6 ms 12:13:51,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 12:13:51,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 12:13:51,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 12:13:51,312 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 12:13:51,314 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 12:13:51,314 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 12:13:51,317 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [07/09/2025 12:13:51.320] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config